{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "参考链接：https://jalammar.github.io/illustrated-bert/    \n",
    "    https://jalammar.github.io/visualizing-neural-machine-translation-mechanics-of-seq2seq-models-with-attention/    \n",
    "        http://jalammar.github.io/illustrated-transformer/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `BERT`基本原理\n",
    "\n",
    "`BERT:Bidirectional Encoder Representations from Transformers`\n",
    "## BERT架构\n",
    "- 自编码语言模型，模型结构为 `Transformer` 的**编码器**；由12层或更多的`EncoderLayer`组成\n",
    "<img src=\"../images/bert架构.png\" width=\"80%\">\n",
    "-----------------                  \n",
    "## BERT预训练                  \n",
    "用两个任务来预训练该模型，如下图 (a) 所示\n",
    "![bert-tasks](../images/bert-tasks.png)\n",
    "- `MaskLM`，输入的话中随机选择一些词，用特殊符号 `[MASK]` 替换，让模型预测这些词\n",
    "    - 训练过程类似“完形填空”。在一句话中随机选择 `15%` 的单词用于预测，其中 `80%` 情况下采用特殊符号 `[MASK]` 替换，`10%` 情况采用任意词替换，`10%` 的情况下保持原词汇不变\n",
    "    - 之所以这样做：在后续微调任务中语句中并不会出现 `[MASK]` 标记\n",
    "    - 预测一个词汇时，模型并不知道输入对应位置的词汇是否为正确的词汇（`10%`概率），这就迫使模型更多地依赖于上下文信息去预测词汇，并且赋予了模型一定的纠错能力。\n",
    "    - 因此模型可能需要更多的训练步骤来收敛    \n",
    "             \n",
    "             \n",
    "- `Next Sentence Prediction`，输入的两句话，让模型判断是否是连续的两句话，让模型学习连续的文本片段之间的关系\n",
    "    - 类似于“连句任务”，给定一篇文章中的两句话，判断第二句话在文本中是否紧跟在第一句话之后\n",
    "    - 文本语料库中随机选择 `50%` 的正确语句对和 `50%` 的错误语句对进行训练\n",
    "\n",
    "\n",
    "  \n",
    "  \n",
    "- 两个任务联合训练，使模型输出的每个字 / 词的向量表示都能尽可能全面、准确地刻画输入文本（单句或语句对）的整体信息，为后续的微调任务提供更好的模型参数初始值。\n",
    "\n",
    "----------------------------------------\n",
    "## BERT微调  \n",
    "如上图 (b) 所示，预训练好的模型，对新的句子进行编码，然后取 `[CLS]`对应的输出向量，进行分类任务；或最后的几个输出向量，进行其它任务\n",
    "\n",
    "------------------------------\n",
    "## BERT优缺点\n",
    "- 相较于 `RNN、LSTM` 可以实现并发训练，同时提取词在句子中的关系特征，在不同的层次提取关系特征，进而更全面反映句子语义。\n",
    "- 相较于 `word2vec`，其又能根据句子上下文获取词义，从而避免歧义出现。\n",
    "- 模型参数太多，而且模型太大，少量数据训练时，容易过拟合。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BERT用于分类任务\n",
    "- 因为bert模型的词汇在预训练时已经固定，所以<span class=\"burk\">必须使用bert模型自带的分词器对文本处理</span>\n",
    "- <span class=\"burk\">文本必须处理成满足bert模型输入的格式要求</span>，添加特殊的`token:[CLS],[SEP],[PAD]`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 语料\n",
    "[`COLA(The Corpus of Linguistic Acceptability)`](https://nyu-mll.github.io/CoLA/)  ,句子在语法上是否可接受`(0=unacceptable, 1=acceptable)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:26:56.936801Z",
     "start_time": "2020-05-05T14:26:56.920375Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "number of training sentences: 8,551\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentence_source</th>\n",
       "      <th>label</th>\n",
       "      <th>label_notes</th>\n",
       "      <th>sentence</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6207</th>\n",
       "      <td>c_13</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>What did Jean think was likely to have been st...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7794</th>\n",
       "      <td>ad03</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Gilgamesh has been fighting the dragon.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6575</th>\n",
       "      <td>g_81</td>\n",
       "      <td>0</td>\n",
       "      <td>*</td>\n",
       "      <td>To which city and which conference did Bill go...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3517</th>\n",
       "      <td>ks08</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>He washed us.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4141</th>\n",
       "      <td>ks08</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>The characters in Shakespeare's Twelfth Night ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     sentence_source  label label_notes  \\\n",
       "6207            c_13      1         NaN   \n",
       "7794            ad03      1         NaN   \n",
       "6575            g_81      0           *   \n",
       "3517            ks08      1         NaN   \n",
       "4141            ks08      1         NaN   \n",
       "\n",
       "                                               sentence  \n",
       "6207  What did Jean think was likely to have been st...  \n",
       "7794            Gilgamesh has been fighting the dragon.  \n",
       "6575  To which city and which conference did Bill go...  \n",
       "3517                                      He washed us.  \n",
       "4141  The characters in Shakespeare's Twelfth Night ...  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv(\"../datasets/cola_public/raw/in_domain_train.tsv\",\n",
    "                 delimiter='\\t',\n",
    "                 header=None,\n",
    "                 names=['sentence_source', 'label', 'label_notes', 'sentence'])\n",
    "\n",
    "print('number of training sentences: {:,}\\n'.format(df.shape[0]))\n",
    "df.sample(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:26:57.127476Z",
     "start_time": "2020-05-05T14:26:57.100292Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentence</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6029</th>\n",
       "      <td>The cake was been eating.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7106</th>\n",
       "      <td>Kim gave a dollar to Bobbie and a dime into hi...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6152</th>\n",
       "      <td>Louis was sent a book to.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5447</th>\n",
       "      <td>He's reliable enough a man.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1769</th>\n",
       "      <td>Willy is taller than Bill is strong and.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               sentence  label\n",
       "6029                          The cake was been eating.      0\n",
       "7106  Kim gave a dollar to Bobbie and a dime into hi...      0\n",
       "6152                          Louis was sent a book to.      0\n",
       "5447                        He's reliable enough a man.      0\n",
       "1769           Willy is taller than Bill is strong and.      0"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc[df.label == 0].sample(5)[['sentence', 'label']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:26:57.375807Z",
     "start_time": "2020-05-05T14:26:57.369341Z"
    }
   },
   "outputs": [],
   "source": [
    "sentences = df.sentence.values\n",
    "labels = df.label.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 分词"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:27:00.011573Z",
     "start_time": "2020-05-05T14:26:59.989521Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Calling BertTokenizer.from_pretrained() with the path to a single file or url is deprecated\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading BERT tokenizer...\n"
     ]
    }
   ],
   "source": [
    "from transformers import BertTokenizer\n",
    "print(\"Loading BERT tokenizer...\")\n",
    "# tokenizer = BertTokenizer.from_pretrained('bert-base-uncased',  # 按模型名称载入\n",
    "#                                           do_lower_case=True)\n",
    "# with open('../models/bert/vocabulary.txt', 'w') as f:\n",
    "#     for token in tokenizer.vocab.keys():\n",
    "#         f.write(token + '\\n')\n",
    "\n",
    "tokenizer = BertTokenizer.from_pretrained(\n",
    "    '../models/bert/vocabulary.txt',  # 从保存有词汇表的本地文件载入\n",
    "    do_lower_case=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:27:06.657043Z",
     "start_time": "2020-05-05T14:27:06.650759Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Original:   Our friends won't buy this analysis, let alone the next one we propose.\n",
      "Tokenized:   ['our', 'friends', 'won', \"'\", 't', 'buy', 'this', 'analysis', ',', 'let', 'alone', 'the', 'next', 'one', 'we', 'propose', '.']\n",
      "Token IDS:   [2256, 2814, 2180, 1005, 1056, 4965, 2023, 4106, 1010, 2292, 2894, 1996, 2279, 2028, 2057, 16599, 1012]\n"
     ]
    }
   ],
   "source": [
    "print(\"Original:  \", sentences[0])\n",
    "print(\"Tokenized:  \", tokenizer.tokenize(sentences[0]))\n",
    "print(\"Token IDS:  \",\n",
    "      tokenizer.convert_tokens_to_ids(tokenizer.tokenize(sentences[0])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 满足`BERT`模型的输入格式"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T12:23:05.299406Z",
     "start_time": "2020-05-05T12:23:05.293885Z"
    }
   },
   "source": [
    "**特殊标记**:\n",
    "- 每个句子的结尾，添加上`[SEP]`标记：用于决定两个句子的关系，如是否是连续的两个句子\n",
    "- 分类任务，在每个句子的开端添加`[CLS]`标记；模型输入的矩阵，只有第一个向量用于分类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**句子长度和遮档**\n",
    "- 语料不同句子长度相差很大，需要所有句子填充或截断到相同的长度；且`BERT`模型，最长512个标记\n",
    "- 填充是添加标记`[PAD]`，在词汇表中索引为 0 \n",
    "    - `[CLS] I like to draw [SEP] [PAD] [PAD]`\n",
    "    \n",
    "- 遮挡是 0 和 1 的序列，1 表示单词，0 表示填充；填充的多少对模型的速度和精度会有影响    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:27:11.016533Z",
     "start_time": "2020-05-05T14:27:10.023850Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Original:  Our friends won't buy this analysis, let alone the next one we propose.\n",
      "Tokenized:   ['our', 'friends', 'won', \"'\", 't', 'buy', 'this', 'analysis', ',', 'let', 'alone', 'the', 'next', 'one', 'we', 'propose', '.']\n",
      "Token IDs:  [101, 2256, 2814, 2180, 1005, 1056, 4965, 2023, 4106, 1010, 2292, 2894, 1996, 2279, 2028, 2057, 16599, 1012, 102]\n"
     ]
    }
   ],
   "source": [
    "# 文本向量化\n",
    "\n",
    "input_ids = []\n",
    "\n",
    "for sent in sentences:\n",
    "    encoded_sent = tokenizer.encode(\n",
    "        sent,\n",
    "        add_special_tokens=True,  # 添加特殊符号\n",
    "    )\n",
    "    input_ids.append(encoded_sent)\n",
    "\n",
    "print(\"Original: \", sentences[0])\n",
    "print(\"Tokenized:  \", tokenizer.tokenize(sentences[0]))\n",
    "print(\"Token IDs: \", input_ids[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:27:11.044124Z",
     "start_time": "2020-05-05T14:27:11.041686Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mac sentence length:  47\n"
     ]
    }
   ],
   "source": [
    "MAX_LEN = max([len(sen) for sen in input_ids])\n",
    "print(\"Max sentence length: \", MAX_LEN)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:27:11.098334Z",
     "start_time": "2020-05-05T14:27:11.068827Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Padding token: [PAD], ID: 0\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(8551, 47)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 填充为相同长度\n",
    "\n",
    "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
    "\n",
    "print(\"Padding token: {:}, ID: {:}\".format(tokenizer.pad_token,\n",
    "                                           tokenizer.pad_token_id))\n",
    "input_ids = pad_sequences(input_ids,\n",
    "                          maxlen=MAX_LEN,\n",
    "                          dtype='long',\n",
    "                          value=0,\n",
    "                          truncating=\"post\",\n",
    "                          padding='post')\n",
    "input_ids.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:27:13.250060Z",
     "start_time": "2020-05-05T14:27:13.097372Z"
    }
   },
   "outputs": [],
   "source": [
    "# 填充对应的mask\n",
    "\n",
    "attention_masks = []\n",
    "for sent in input_ids:\n",
    "    att_mask = [int(token_id > 0) for token_id in sent]\n",
    "    attention_masks.append(att_mask)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:27:13.295838Z",
     "start_time": "2020-05-05T14:27:13.288722Z"
    }
   },
   "outputs": [],
   "source": [
    "# 拆分为测试集和训练集\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "train_inputs, validation_inputs, train_labels, validation_labels = train_test_split(\n",
    "    input_ids, labels, random_state=2008, test_size=0.1)\n",
    "\n",
    "train_masks, validation_masks, _, _ = train_test_split(attention_masks,\n",
    "                                                        labels,\n",
    "                                                        random_state=2008,\n",
    "                                                        test_size=0.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:27:13.430033Z",
     "start_time": "2020-05-05T14:27:13.400825Z"
    }
   },
   "outputs": [],
   "source": [
    "# 转化为 PyTorch 数据格式\n",
    "import torch\n",
    "\n",
    "train_inputs = torch.tensor(train_inputs)\n",
    "validation_inputs = torch.tensor(validation_inputs)\n",
    "\n",
    "train_labels = torch.tensor(train_labels)\n",
    "validation_labels = torch.tensor(validation_labels)\n",
    "\n",
    "train_masks = torch.tensor(train_masks)\n",
    "validation_masks = torch.tensor(validation_masks)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:27:16.274713Z",
     "start_time": "2020-05-05T14:27:16.268166Z"
    }
   },
   "outputs": [],
   "source": [
    "# 创建数据管道\n",
    "\n",
    "from torch.utils.data import TensorDataset, DataLoader, RandomSampler, SequentialSampler\n",
    "\n",
    "batch_size = 32\n",
    "train_data = TensorDataset(train_inputs, train_masks, train_labels)\n",
    "train_sampler = RandomSampler(train_data)\n",
    "train_dataloader = DataLoader(train_data,\n",
    "                          sampler=train_sampler,\n",
    "                          batch_size=batch_size)\n",
    "\n",
    "validation_data = TensorDataset(validation_inputs, validation_masks,\n",
    "                                validation_labels)\n",
    "validation_sampler = RandomSampler(validation_data)\n",
    "validation_dataloader = DataLoader(validation_data,\n",
    "                                   sampler=validation_sampler,\n",
    "                                   batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 创建并训练分类模型\n",
    "`huggingface`提供的接口：\n",
    "\n",
    "- BertModel\n",
    "- BertForPreTraining\n",
    "- BertForMaskedLM\n",
    "- BertForNextSentencePrediction\n",
    "- **BertForSequenceClassification** \n",
    "- BertForTokenClassification\n",
    "- BertForQuestionAnswering    \n",
    "  \n",
    "将模型`\"bert-base-uncased\": \"https://cdn.huggingface.co/bert-base-uncased-pytorch_model.bin\"`,  \n",
    "模型词汇表`\"bert-base-uncased\": \"https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-vocab.txt\",`,  \n",
    "模型配置`\"bert-base-uncased\": \"https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-config.json\",`三个文件下载到本地，文件夹命名为`\"bert-base-uncased\"`, 然后更改文件名，删除名字中的前缀`\"bert-base-uncased\"`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:29:57.571668Z",
     "start_time": "2020-05-05T14:29:56.321464Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BertForSequenceClassification(\n",
       "  (bert): BertModel(\n",
       "    (embeddings): BertEmbeddings(\n",
       "      (word_embeddings): Embedding(30522, 768, padding_idx=0)\n",
       "      (position_embeddings): Embedding(512, 768)\n",
       "      (token_type_embeddings): Embedding(2, 768)\n",
       "      (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "      (dropout): Dropout(p=0.1, inplace=False)\n",
       "    )\n",
       "    (encoder): BertEncoder(\n",
       "      (layer): ModuleList(\n",
       "        (0): BertLayer(\n",
       "          (attention): BertAttention(\n",
       "            (self): BertSelfAttention(\n",
       "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "            (output): BertSelfOutput(\n",
       "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "          )\n",
       "          (intermediate): BertIntermediate(\n",
       "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
       "          )\n",
       "          (output): BertOutput(\n",
       "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
       "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "            (dropout): Dropout(p=0.1, inplace=False)\n",
       "          )\n",
       "        )\n",
       "        (1): BertLayer(\n",
       "          (attention): BertAttention(\n",
       "            (self): BertSelfAttention(\n",
       "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "            (output): BertSelfOutput(\n",
       "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "          )\n",
       "          (intermediate): BertIntermediate(\n",
       "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
       "          )\n",
       "          (output): BertOutput(\n",
       "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
       "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "            (dropout): Dropout(p=0.1, inplace=False)\n",
       "          )\n",
       "        )\n",
       "        (2): BertLayer(\n",
       "          (attention): BertAttention(\n",
       "            (self): BertSelfAttention(\n",
       "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "            (output): BertSelfOutput(\n",
       "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "          )\n",
       "          (intermediate): BertIntermediate(\n",
       "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
       "          )\n",
       "          (output): BertOutput(\n",
       "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
       "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "            (dropout): Dropout(p=0.1, inplace=False)\n",
       "          )\n",
       "        )\n",
       "        (3): BertLayer(\n",
       "          (attention): BertAttention(\n",
       "            (self): BertSelfAttention(\n",
       "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "            (output): BertSelfOutput(\n",
       "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "          )\n",
       "          (intermediate): BertIntermediate(\n",
       "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
       "          )\n",
       "          (output): BertOutput(\n",
       "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
       "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "            (dropout): Dropout(p=0.1, inplace=False)\n",
       "          )\n",
       "        )\n",
       "        (4): BertLayer(\n",
       "          (attention): BertAttention(\n",
       "            (self): BertSelfAttention(\n",
       "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "            (output): BertSelfOutput(\n",
       "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "          )\n",
       "          (intermediate): BertIntermediate(\n",
       "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
       "          )\n",
       "          (output): BertOutput(\n",
       "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
       "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "            (dropout): Dropout(p=0.1, inplace=False)\n",
       "          )\n",
       "        )\n",
       "        (5): BertLayer(\n",
       "          (attention): BertAttention(\n",
       "            (self): BertSelfAttention(\n",
       "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "            (output): BertSelfOutput(\n",
       "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "          )\n",
       "          (intermediate): BertIntermediate(\n",
       "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
       "          )\n",
       "          (output): BertOutput(\n",
       "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
       "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "            (dropout): Dropout(p=0.1, inplace=False)\n",
       "          )\n",
       "        )\n",
       "        (6): BertLayer(\n",
       "          (attention): BertAttention(\n",
       "            (self): BertSelfAttention(\n",
       "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "            (output): BertSelfOutput(\n",
       "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "          )\n",
       "          (intermediate): BertIntermediate(\n",
       "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
       "          )\n",
       "          (output): BertOutput(\n",
       "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
       "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "            (dropout): Dropout(p=0.1, inplace=False)\n",
       "          )\n",
       "        )\n",
       "        (7): BertLayer(\n",
       "          (attention): BertAttention(\n",
       "            (self): BertSelfAttention(\n",
       "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "            (output): BertSelfOutput(\n",
       "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "          )\n",
       "          (intermediate): BertIntermediate(\n",
       "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
       "          )\n",
       "          (output): BertOutput(\n",
       "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
       "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "            (dropout): Dropout(p=0.1, inplace=False)\n",
       "          )\n",
       "        )\n",
       "        (8): BertLayer(\n",
       "          (attention): BertAttention(\n",
       "            (self): BertSelfAttention(\n",
       "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "            (output): BertSelfOutput(\n",
       "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "          )\n",
       "          (intermediate): BertIntermediate(\n",
       "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
       "          )\n",
       "          (output): BertOutput(\n",
       "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
       "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "            (dropout): Dropout(p=0.1, inplace=False)\n",
       "          )\n",
       "        )\n",
       "        (9): BertLayer(\n",
       "          (attention): BertAttention(\n",
       "            (self): BertSelfAttention(\n",
       "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "            (output): BertSelfOutput(\n",
       "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "          )\n",
       "          (intermediate): BertIntermediate(\n",
       "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
       "          )\n",
       "          (output): BertOutput(\n",
       "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
       "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "            (dropout): Dropout(p=0.1, inplace=False)\n",
       "          )\n",
       "        )\n",
       "        (10): BertLayer(\n",
       "          (attention): BertAttention(\n",
       "            (self): BertSelfAttention(\n",
       "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "            (output): BertSelfOutput(\n",
       "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "          )\n",
       "          (intermediate): BertIntermediate(\n",
       "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
       "          )\n",
       "          (output): BertOutput(\n",
       "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
       "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "            (dropout): Dropout(p=0.1, inplace=False)\n",
       "          )\n",
       "        )\n",
       "        (11): BertLayer(\n",
       "          (attention): BertAttention(\n",
       "            (self): BertSelfAttention(\n",
       "              (query): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (key): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (value): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "            (output): BertSelfOutput(\n",
       "              (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "              (dropout): Dropout(p=0.1, inplace=False)\n",
       "            )\n",
       "          )\n",
       "          (intermediate): BertIntermediate(\n",
       "            (dense): Linear(in_features=768, out_features=3072, bias=True)\n",
       "          )\n",
       "          (output): BertOutput(\n",
       "            (dense): Linear(in_features=3072, out_features=768, bias=True)\n",
       "            (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "            (dropout): Dropout(p=0.1, inplace=False)\n",
       "          )\n",
       "        )\n",
       "      )\n",
       "    )\n",
       "    (pooler): BertPooler(\n",
       "      (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "      (activation): Tanh()\n",
       "    )\n",
       "  )\n",
       "  (dropout): Dropout(p=0.1, inplace=False)\n",
       "  (classifier): Linear(in_features=768, out_features=2, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from transformers import BertForSequenceClassification, AdamW, BertConfig\n",
    "import os\n",
    "\n",
    "# 载入 bert 预训练模型\n",
    "\n",
    "path = os.path.abspath(\"../../H/models/huggingface/bert-base-uncased/\")\n",
    "\n",
    "model = BertForSequenceClassification.from_pretrained(\n",
    "    # \"bert-base-uncased\",   # 按名称载入模型\n",
    "    path,  # 本地文件载入\n",
    "    num_labels=2,\n",
    "    output_attentions=False,\n",
    "    output_hidden_states=False,\n",
    ")\n",
    "\n",
    "model.cuda()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:30:37.493775Z",
     "start_time": "2020-05-05T14:30:37.484955Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The BERT model has 201 different named parameters.\n",
      "\n",
      "==== Embedding Layer ====\n",
      "\n",
      "bert.embeddings.word_embeddings.weight                  (30522, 768)\n",
      "bert.embeddings.position_embeddings.weight                (512, 768)\n",
      "bert.embeddings.token_type_embeddings.weight                (2, 768)\n",
      "bert.embeddings.LayerNorm.weight                              (768,)\n",
      "bert.embeddings.LayerNorm.bias                                (768,)\n",
      "\n",
      "==== First Transformer ====\n",
      "\n",
      "bert.encoder.layer.0.attention.self.query.weight          (768, 768)\n",
      "bert.encoder.layer.0.attention.self.query.bias                (768,)\n",
      "bert.encoder.layer.0.attention.self.key.weight            (768, 768)\n",
      "bert.encoder.layer.0.attention.self.key.bias                  (768,)\n",
      "bert.encoder.layer.0.attention.self.value.weight          (768, 768)\n",
      "bert.encoder.layer.0.attention.self.value.bias                (768,)\n",
      "bert.encoder.layer.0.attention.output.dense.weight        (768, 768)\n",
      "bert.encoder.layer.0.attention.output.dense.bias              (768,)\n",
      "bert.encoder.layer.0.attention.output.LayerNorm.weight        (768,)\n",
      "bert.encoder.layer.0.attention.output.LayerNorm.bias          (768,)\n",
      "bert.encoder.layer.0.intermediate.dense.weight           (3072, 768)\n",
      "bert.encoder.layer.0.intermediate.dense.bias                 (3072,)\n",
      "bert.encoder.layer.0.output.dense.weight                 (768, 3072)\n",
      "bert.encoder.layer.0.output.dense.bias                        (768,)\n",
      "bert.encoder.layer.0.output.LayerNorm.weight                  (768,)\n",
      "bert.encoder.layer.0.output.LayerNorm.bias                    (768,)\n",
      "\n",
      "==== Output Layer ====\n",
      "\n",
      "bert.pooler.dense.weight                                  (768, 768)\n",
      "bert.pooler.dense.bias                                        (768,)\n",
      "classifier.weight                                           (2, 768)\n",
      "classifier.bias                                                 (2,)\n"
     ]
    }
   ],
   "source": [
    "# 模型参数详情\n",
    "\n",
    "params = list(model.named_parameters())\n",
    "\n",
    "print('The BERT model has {:} different named parameters.\\n'.format(\n",
    "    len(params)))\n",
    "\n",
    "print('==== Embedding Layer ====\\n')\n",
    "\n",
    "for p in params[0:5]:\n",
    "    print(\"{:<55} {:>12}\".format(p[0], str(tuple(p[1].size()))))\n",
    "\n",
    "print('\\n==== First Transformer ====\\n')\n",
    "\n",
    "for p in params[5:21]:\n",
    "    print(\"{:<55} {:>12}\".format(p[0], str(tuple(p[1].size()))))\n",
    "\n",
    "print('\\n==== Output Layer ====\\n')\n",
    "\n",
    "for p in params[-4:]:\n",
    "    print(\"{:<55} {:>12}\".format(p[0], str(tuple(p[1].size()))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:33:27.887434Z",
     "start_time": "2020-05-05T14:33:27.882112Z"
    }
   },
   "outputs": [],
   "source": [
    "# 优化器\n",
    "optimizer = AdamW(model.parameters(), lr=2e-5, eps=1e-8)\n",
    "\n",
    "\n",
    "# 学习率规划\n",
    "from transformers import get_linear_schedule_with_warmup\n",
    "\n",
    "epochs = 4\n",
    "\n",
    "total_steps = len(train_dataloader) * epochs\n",
    "\n",
    "scheduler = get_linear_schedule_with_warmup(optimizer,\n",
    "                                            num_warmup_steps=0,\n",
    "                                            num_training_steps=total_steps)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:34:13.043466Z",
     "start_time": "2020-05-05T14:34:13.038266Z"
    }
   },
   "outputs": [],
   "source": [
    "\n",
    "# 预测精度\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "def flat_accuracy(preds, labels):\n",
    "    pred_flat = np.argmax(preds, axis=1).flatten()\n",
    "    labels_flat = labels.flatten()\n",
    "    return np.sum(pred_flat == labels_flat) / len(labels_flat)\n",
    "\n",
    "\n",
    "# 格式化时间显示\n",
    "import time\n",
    "import datetime\n",
    "\n",
    "\n",
    "def format_time(elapsed):\n",
    "    elapsed_rounded = int(round((elapsed)))\n",
    "\n",
    "    return str(datetime.timedelta(seconds=elapsed_rounded))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:40:13.289241Z",
     "start_time": "2020-05-05T14:38:17.161133Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "======== Epoch 1 / 4 ========\n",
      "Training...\n",
      "  Batch    40  of    241.    Elapsed: 0:00:05.\n",
      "  Batch    80  of    241.    Elapsed: 0:00:09.\n",
      "  Batch   120  of    241.    Elapsed: 0:00:14.\n",
      "  Batch   160  of    241.    Elapsed: 0:00:19.\n",
      "  Batch   200  of    241.    Elapsed: 0:00:23.\n",
      "  Batch   240  of    241.    Elapsed: 0:00:28.\n",
      "\n",
      "  Average training loss: 0.17\n",
      "  Training epcoh took: 0:00:28\n",
      "\n",
      "Running Validation...\n",
      "  Accuracy: 0.80\n",
      "  Validation took: 0:00:01\n",
      "\n",
      "======== Epoch 2 / 4 ========\n",
      "Training...\n",
      "  Batch    40  of    241.    Elapsed: 0:00:05.\n",
      "  Batch    80  of    241.    Elapsed: 0:00:09.\n",
      "  Batch   120  of    241.    Elapsed: 0:00:14.\n",
      "  Batch   160  of    241.    Elapsed: 0:00:19.\n",
      "  Batch   200  of    241.    Elapsed: 0:00:23.\n",
      "  Batch   240  of    241.    Elapsed: 0:00:28.\n",
      "\n",
      "  Average training loss: 0.20\n",
      "  Training epcoh took: 0:00:28\n",
      "\n",
      "Running Validation...\n",
      "  Accuracy: 0.81\n",
      "  Validation took: 0:00:01\n",
      "\n",
      "======== Epoch 3 / 4 ========\n",
      "Training...\n",
      "  Batch    40  of    241.    Elapsed: 0:00:05.\n",
      "  Batch    80  of    241.    Elapsed: 0:00:09.\n",
      "  Batch   120  of    241.    Elapsed: 0:00:14.\n",
      "  Batch   160  of    241.    Elapsed: 0:00:19.\n",
      "  Batch   200  of    241.    Elapsed: 0:00:23.\n",
      "  Batch   240  of    241.    Elapsed: 0:00:28.\n",
      "\n",
      "  Average training loss: 0.13\n",
      "  Training epcoh took: 0:00:28\n",
      "\n",
      "Running Validation...\n",
      "  Accuracy: 0.82\n",
      "  Validation took: 0:00:01\n",
      "\n",
      "======== Epoch 4 / 4 ========\n",
      "Training...\n",
      "  Batch    40  of    241.    Elapsed: 0:00:05.\n",
      "  Batch    80  of    241.    Elapsed: 0:00:09.\n",
      "  Batch   120  of    241.    Elapsed: 0:00:14.\n",
      "  Batch   160  of    241.    Elapsed: 0:00:19.\n",
      "  Batch   200  of    241.    Elapsed: 0:00:23.\n",
      "  Batch   240  of    241.    Elapsed: 0:00:28.\n",
      "\n",
      "  Average training loss: 0.12\n",
      "  Training epcoh took: 0:00:28\n",
      "\n",
      "Running Validation...\n",
      "  Accuracy: 0.82\n",
      "  Validation took: 0:00:01\n",
      "\n",
      "Training complete!\n"
     ]
    }
   ],
   "source": [
    "# 训练并验证模型\n",
    "\n",
    "import random\n",
    "\n",
    "# https://github.com/huggingface/transformers/blob/5bfcd0485ece086ebcbed2d008813037968a9e58/examples/run_glue.py#L128\n",
    "\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "# Set the seed value all over the place to make this reproducible.\n",
    "seed_val = 42\n",
    "\n",
    "random.seed(seed_val)\n",
    "np.random.seed(seed_val)\n",
    "torch.manual_seed(seed_val)\n",
    "torch.cuda.manual_seed_all(seed_val)\n",
    "\n",
    "# Store the average loss after each epoch so we can plot them.\n",
    "loss_values = []\n",
    "\n",
    "# For each epoch...\n",
    "for epoch_i in range(0, epochs):\n",
    "\n",
    "    # ========================================\n",
    "    #               Training\n",
    "    # ========================================\n",
    "\n",
    "    # Perform one full pass over the training set.\n",
    "\n",
    "    print(\"\")\n",
    "    print('======== Epoch {:} / {:} ========'.format(epoch_i + 1, epochs))\n",
    "    print('Training...')\n",
    "\n",
    "    # Measure how long the training epoch takes.\n",
    "    t0 = time.time()\n",
    "\n",
    "    # Reset the total loss for this epoch.\n",
    "    total_loss = 0\n",
    "\n",
    "    # Put the model into training mode. Don't be mislead--the call to\n",
    "    # `train` just changes the *mode*, it doesn't *perform* the training.\n",
    "    # `dropout` and `batchnorm` layers behave differently during training\n",
    "    # vs. test (source: https://stackoverflow.com/questions/51433378/what-does-model-train-do-in-pytorch)\n",
    "    model.train()\n",
    "\n",
    "    # For each batch of training data...\n",
    "    for step, batch in enumerate(train_dataloader):\n",
    "\n",
    "        # Progress update every 40 batches.\n",
    "        if step % 40 == 0 and not step == 0:\n",
    "            # Calculate elapsed time in minutes.\n",
    "            elapsed = format_time(time.time() - t0)\n",
    "\n",
    "            # Report progress.\n",
    "            print('  Batch {:>5,}  of  {:>5,}.    Elapsed: {:}.'.format(\n",
    "                step, len(train_dataloader), elapsed))\n",
    "\n",
    "        # Unpack this training batch from our dataloader.\n",
    "        #\n",
    "        # As we unpack the batch, we'll also copy each tensor to the GPU using the\n",
    "        # `to` method.\n",
    "        #\n",
    "        # `batch` contains three pytorch tensors:\n",
    "        #   [0]: input ids\n",
    "        #   [1]: attention masks\n",
    "        #   [2]: labels\n",
    "        b_input_ids = batch[0].to(device)\n",
    "        b_input_mask = batch[1].to(device)\n",
    "        b_labels = batch[2].to(device)\n",
    "\n",
    "        # Always clear any previously calculated gradients before performing a\n",
    "        # backward pass. PyTorch doesn't do this automatically because\n",
    "        # accumulating the gradients is \"convenient while training RNNs\".\n",
    "        # (source: https://stackoverflow.com/questions/48001598/why-do-we-need-to-call-zero-grad-in-pytorch)\n",
    "        model.zero_grad()\n",
    "\n",
    "        # Perform a forward pass (evaluate the model on this training batch).\n",
    "        # This will return the loss (rather than the model output) because we\n",
    "        # have provided the `labels`.\n",
    "        # The documentation for this `model` function is here:\n",
    "        # https://huggingface.co/transformers/v2.2.0/model_doc/bert.html#transformers.BertForSequenceClassification\n",
    "        outputs = model(b_input_ids,\n",
    "                        token_type_ids=None,\n",
    "                        attention_mask=b_input_mask,\n",
    "                        labels=b_labels)\n",
    "\n",
    "        # The call to `model` always returns a tuple, so we need to pull the\n",
    "        # loss value out of the tuple.\n",
    "        loss = outputs[0]\n",
    "\n",
    "        # Accumulate the training loss over all of the batches so that we can\n",
    "        # calculate the average loss at the end. `loss` is a Tensor containing a\n",
    "        # single value; the `.item()` function just returns the Python value\n",
    "        # from the tensor.\n",
    "        total_loss += loss.item()\n",
    "\n",
    "        # Perform a backward pass to calculate the gradients.\n",
    "        loss.backward()\n",
    "\n",
    "        # Clip the norm of the gradients to 1.0.\n",
    "        # This is to help prevent the \"exploding gradients\" problem.\n",
    "        torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)\n",
    "\n",
    "        # Update parameters and take a step using the computed gradient.\n",
    "        # The optimizer dictates the \"update rule\"--how the parameters are\n",
    "        # modified based on their gradients, the learning rate, etc.\n",
    "        optimizer.step()\n",
    "\n",
    "        # Update the learning rate.\n",
    "        scheduler.step()\n",
    "\n",
    "    # Calculate the average loss over the training data.\n",
    "    avg_train_loss = total_loss / len(train_dataloader)\n",
    "\n",
    "    # Store the loss value for plotting the learning curve.\n",
    "    loss_values.append(avg_train_loss)\n",
    "\n",
    "    print(\"\")\n",
    "    print(\"  Average training loss: {0:.2f}\".format(avg_train_loss))\n",
    "    print(\"  Training epcoh took: {:}\".format(format_time(time.time() - t0)))\n",
    "\n",
    "    # ========================================\n",
    "    #               Validation\n",
    "    # ========================================\n",
    "    # After the completion of each training epoch, measure our performance on\n",
    "    # our validation set.\n",
    "\n",
    "    print(\"\")\n",
    "    print(\"Running Validation...\")\n",
    "\n",
    "    t0 = time.time()\n",
    "\n",
    "    # Put the model in evaluation mode--the dropout layers behave differently\n",
    "    # during evaluation.\n",
    "    model.eval()\n",
    "\n",
    "    # Tracking variables\n",
    "    eval_loss, eval_accuracy = 0, 0\n",
    "    nb_eval_steps, nb_eval_examples = 0, 0\n",
    "\n",
    "    # Evaluate data for one epoch\n",
    "    for batch in validation_dataloader:\n",
    "\n",
    "        # Add batch to GPU\n",
    "        batch = tuple(t.to(device) for t in batch)\n",
    "\n",
    "        # Unpack the inputs from our dataloader\n",
    "        b_input_ids, b_input_mask, b_labels = batch\n",
    "\n",
    "        # Telling the model not to compute or store gradients, saving memory and\n",
    "        # speeding up validation\n",
    "        with torch.no_grad():\n",
    "\n",
    "            # Forward pass, calculate logit predictions.\n",
    "            # This will return the logits rather than the loss because we have\n",
    "            # not provided labels.\n",
    "            # token_type_ids is the same as the \"segment ids\", which\n",
    "            # differentiates sentence 1 and 2 in 2-sentence tasks.\n",
    "            # The documentation for this `model` function is here:\n",
    "            # https://huggingface.co/transformers/v2.2.0/model_doc/bert.html#transformers.BertForSequenceClassification\n",
    "            outputs = model(b_input_ids,\n",
    "                            token_type_ids=None,\n",
    "                            attention_mask=b_input_mask)\n",
    "\n",
    "        # Get the \"logits\" output by the model. The \"logits\" are the output\n",
    "        # values prior to applying an activation function like the softmax.\n",
    "        logits = outputs[0]\n",
    "\n",
    "        # Move logits and labels to CPU\n",
    "        logits = logits.detach().cpu().numpy()\n",
    "        label_ids = b_labels.to('cpu').numpy()\n",
    "\n",
    "        # Calculate the accuracy for this batch of test sentences.\n",
    "        tmp_eval_accuracy = flat_accuracy(logits, label_ids)\n",
    "\n",
    "        # Accumulate the total accuracy.\n",
    "        eval_accuracy += tmp_eval_accuracy\n",
    "\n",
    "        # Track the number of batches\n",
    "        nb_eval_steps += 1\n",
    "\n",
    "    # Report the final accuracy for this validation run.\n",
    "    print(\"  Accuracy: {0:.2f}\".format(eval_accuracy / nb_eval_steps))\n",
    "    print(\"  Validation took: {:}\".format(format_time(time.time() - t0)))\n",
    "\n",
    "print(\"\")\n",
    "print(\"Training complete!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:40:26.343611Z",
     "start_time": "2020-05-05T14:40:26.185696Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvAAAAGXCAYAAADVv2QFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdZ0CUZ9o+/GPoIEOVoiBFB1AROwI2LKAJxoLdWFPsJLp5stndx8TNk8QE89+VfSMSW4omGjWi2BIURbGBKEYFMXQpdmlDkzbzfjCQsIChDNz3wPH7Ntfc5RzOEM65vM77kiiVSiWIiIiIiEgtaAgdABERERERNR0LeCIiIiIiNcICnoiIiIhIjbCAJyIiIiJSIyzgiYiIiIjUCAt4IiIiIiI1wgKeiKiT+te//gUXFxc8efKkReeXl5fDxcUF69evV3FkzfPDDz/AxcUFN27cEDQOIqL2oiV0AEREnZmLi0uTjz1z5gxsbW3bMBoiIlIHLOCJiAT0+eef13kdFxeH/fv3Y86cORgyZEid98zMzFR677Vr1+Ktt96Crq5ui87X1dXFrVu3oKmpqdK4iIjoxVjAExEJaOrUqXVeV1dXY//+/Rg4cGC99xqjVCpRVlYGAwODZt1bS0sLWlqt+zPQ0uKfiIhajmvgiYjUyPnz5+Hi4oLjx49j165deOmll+Dm5obvv/8eAHD9+nW89957mDBhAgYMGIDBgwdj/vz5OHv2bL1rNbQGvmYsOzsbGzduxKhRo+Dm5gZ/f39cunSpzvkNrYH/49jVq1cxb948DBgwAJ6enli/fj3KysrqxXH58mXMmjULbm5uGDlyJDZu3IjExES4uLhg+/btLf5ZPX36FOvXr8fo0aPRr18/jB07Fp988gkKCwvrHFdaWoqgoCBMnDgR/fv3h7u7OyZPnoygoKA6x50+fRrz5s2Dh4cH+vfvj7Fjx+Ltt99GdnZ2i2MkImoJzsATEamhHTt2oKioCDNmzIC5uTl69OgBAAgPD0dWVhb8/PzQvXt35OXl4fDhw1ixYgU2b96MCRMmNOn6//M//wNdXV28+eabKC8vx7fffouVK1ciIiICVlZWf3p+fHw8Tp48iZkzZ2LKlCmIjo7G/v37oaOjg/fff7/2uOjoaCxduhRmZmZYvnw5DA0NceLECVy5cqVlP5jfFBQUYM6cObh//z5mzZqF3r17Iz4+Ht9//z2uXLmCAwcOQF9fHwDwwQcf4MSJE/D398fAgQNRWVmJu3fvIiYmpvZ6Fy9eREBAAPr27YsVK1bA0NAQjx49wqVLl5CTk1P78yciag8s4ImI1NDjx4/x888/w8TEpM742rVr6y2lWbhwIaZMmYIvv/yyyQW8lZUVvvjiC0gkEgConcn/8ccfERAQ8KfnJyUl4eDBg+jbty8AYN68eVi8eDH279+P9957Dzo6OgCAzz77DNra2jhw4AC6desGAHj11Vcxd+7cJsXZmK1btyInJwcbNmzAzJkza8ednJywcePG2i8kSqUSkZGR8PHxwWeffdbo9U6fPg0A2LVrF6RSae14U34WRESqxiU0RERqaMaMGfWKdwB1iveysjLk5+ejvLwcw4YNw507d1BRUdGk6y9evLi2eAeAIUOGQFtbG3fv3m3S+e7u7rXFew1PT09UVFTgwYMHAIB79+4hKSkJEydOrC3eAUBHRweLFi1q0n0aU/MvBdOnT68zvmDBAkilUkRERAAAJBIJunTpgqSkJKSlpTV6PalUCqVSiZMnT6K6urpVsRERtRZn4ImI1JCDg0OD448fP0ZQUBDOnj2L/Pz8eu8XFRXB3Nz8T6//30tCJBIJjI2NUVBQ0KT4GlpSUvOFo6CgAPb29sjJyQEAODo61ju2obGmUiqVuH//Pjw9PaGhUXeeSkdHB3Z2drX3BoB169bhf//3f+Hn5wd7e3t4eHhg3LhxGDNmTO2XmMWLF+PcuXNYt24dAgMDMXToUIwaNQp+fn4wNTVtcaxERC3BAp6ISA3VrN/+o+rqaixZsgQ5OTlYtGgRXF1dIZVKoaGhgX379uHkyZNQKBRNuv5/F741lEplq87/4zWaeq3mau51X375ZXh4eOD8+fOIjY3FxYsXceDAAXh5eWHnzp3Q0tJC165dcfjwYVy9ehWXL1/G1atX8cknn+CLL77AV199hX79+rXJZyEiaggLeCKiDiIhIQFpaWl45513sHz58jrv1TylRkxqNqXKyMio915DY02loaEBGxsbpKenQ6FQ1PkyUVFRgaysLNjZ2dU5x8zMDNOmTcO0adOgVCrx6aefYvfu3Th//jzGjRsH4PljN728vODl5QXg+c975syZ2LZtGzZv3tzieImImotr4ImIOoiaQvW/Z6Bv376NqKgoIUJ6IVtbWzg7O+PkyZO16+KB50X27t27W3VtHx8fPHz4EGFhYXXG9+7di6KiIvj6+gIAKisrUVxcXOcYiUSCPn36AEDtIyfz8vLq3UMmk0FHR6fJy4qIiFSFM/BERB2Ei4sLHBwc8OWXX0Iul8PBwQFpaWk4cOAAXFxccPv2baFDrOfvf/87li5ditmzZ2Pu3Lno0qULTpw4UaeBtiVWrFiBU6dO4f3338fNmzfh4uKChIQEHDp0CM7OzliyZAmA5+vxfXx84OPjAxcXF5iZmSE7Oxs//PADTE1N4e3tDQB47733IJfL4eXlBRsbG5SWluL48eMoLy/HtGnTWvtjICJqFhbwREQdhI6ODnbs2IHPP/8coaGhKC8vh7OzMzZt2oS4uDhRFvAjRozAtm3b8J///Adbt26FsbExXnnlFfj4+GD+/PnQ09Nr0XVNTEywf/9+bN68GWfOnEFoaCjMzc2xYMECvPXWW7U9BFKpFAsWLEB0dDQuXLiAsrIyWFhYYMKECVi+fDnMzMwAANOnT8eRI0dw6NAh5OfnQyqVwsnJCSEhIRg/frzKfh5ERE0hUbZVFxEREVELHT16FH/961+xZcsW+Pj4CB0OEZGocA08EREJRqFQ1Hs2fUVFBXbt2gUdHR0MGTJEoMiIiMSLS2iIiEgwxcXF8PPzw+TJk+Hg4IC8vDycOHECKSkpCAgI4DPWiYgawAKeiIgEo6enhxEjRuDUqVN4+vQpAKBnz5746KOPMGfOHIGjIyISJ66BJyIiIiJSI1wDT0RERESkRljAExERERGpEa6Bb6b8/BIoFO2/6sjc3BC5ucV/fiC1G+ZEnJgX8WFOxIl5ER/mRJyEyIuGhgSmpl0afV/QAr6kpARBQUEIDw+HXC6HTCbD6tWr/3RTjB9//BFnzpxBUlIScnNzYW1tjdGjR2PVqlW1m2780e7du7Fnzx7cu3cP1tbWmDNnDt54443abcebQ6FQClLA19ybxIU5ESfmRXyYE3FiXsSHOREnseVF0CU0AQEBOHbsGNasWYNt27ZBJpMhICAAUVFRLzzviy++gKGhId555x3s3LkTS5Yswc8//4yZM2dCLpfXOTYkJASfffYZ/Pz88NVXX2HmzJn4z3/+g02bNrXlRyMiIiIiahOCzcBHRUXh8uXLCA4Ohq+vLwDA09MT2dnZCAwMhLe3d6PnhoWFwdzcvPb1sGHDIJPJsHDhQhw5cgQLFy4EAOTn52Pr1q2YP38+1qxZAwDw8PBAWVkZdu7ciQULFsDa2roNPyURERERkWoJNgMfEREBqVRaZ7mMRCKBv78/0tPTkZqa2ui5fyzea7i5uQEAHj58WDt24cIFlJeXw9/fv86x/v7+qKqqwpkzZ1r7MYiIiIiI2pVgBXxKSgpkMlm9deguLi4AgOTk5GZdLyYmBgDg5ORU5x4SiaTOGAA4ODhAT08PKSkpLQmdiIiIiEgwgi2hKSgogIODQ71xY2Pj2vebc61PPvkEDg4O8PPzqzOur68PHR2deucYGRk16x41zM0Nm32OqlhYSAW7NzWMOREn5kV8mBNxYl7EhzkRJ7HlRdCn0Egkkha990dlZWVYvXo1CgsL8f333zdYrLf2Hn+Um1ssSCeyhYUUT54Utft9qXHMiTgxL+LDnIgT8yI+zIk4CZEXDQ3JCyeNBVtCY2Ji0uAMeGFhIYDfZ+Jf5NmzZ1i5ciUSExOxfft29O7du949ysrKUFFRUe9cuVzepHsQEREREYmJYAW8TCZDWloaFApFnfGate/Ozs4vPL+8vByrVq3CjRs3sG3bNgwePLjBeyiVynpr3TMzM/Hs2bN6a+OJiIiIiMROsALe19cXcrkckZGRdcbDwsLg6OgImUzW6LkVFRVYtWoVrl27hpCQEAwbNqzB40aPHg0dHR0cOXKkzvjhw4ehpaWFcePGtf6DEBERERG1I8HWwHt7e8PDwwPr1q1DQUEBbG1tERYWhri4OISEhNQet3DhQsTGxiIpKal27O2338bFixexevVqGBgY4MaNG7XvmZmZwc7ODgBgamqK5cuXIyQkBFKpFB4eHrhx4wZ27tyJRYsWoVu3bu33gYmoTUXffohDUWnIk5fDzEgX0717wcuV+zwQEVHHI1EqlYLtDVtcXIxNmzbh5MmTkMvlkMlkWL16NXx8fGqPaaiAr3nUZEP8/f0RGBhY+1qpVGLXrl3Yu3cv7t+/D0tLS8yZMwdLly6t9wjLpmATK9VgTsQj+vZD7Pr5V1RU/b4kT0dLA4tf7s0iXgT4uyJOzIv4MCfiJMYmVkELeHXEAp5qMCfi8deQS8iVl9cbNzfSxf9bNUKAiOiP+LsiTsyL+DAn4iTGAl6wNfBERKpQ+qyqweIdQKPjRERE6kzQ58ATEbWUQqlEdMJD/HgurdFjzI102zEiIiKi9sECnojUTubDInwfkYS0e3L06m6EcYNt8FN0Zp018BIJMG1UTwGjJCIiahss4IlIbRSVVuDQ+XScv3EfUgNtvO7XB8PdrKEhkcDCRL/2KTRd9LVRXFaJ9PtyjHDj06aIiKhjYQFPRKKnUChx7sY9HD6fjrLyavi698CUEY4w0Pv9f2FertbwcrWubTb68Wwqfr6SBXtrKUYP6C5g9ERERKrFAp6IRC05uwB7IpKR/bgYfexN8aqPE2wsGu/MrzHDuxeyHhfj+1NJsOnaBb1sjNshWiIiorbHp9AQkSjlF5Vj+7HbCNxzHSXPKrFqWj+8O3dgk4p34PkjuJZPcYWpVBdbDsejsJhPpCEioo6BBTwRiUpVtQI/X8nE/+6IwbVfn+CV4Q7Y8KYnhva2hEQiada1DPW18db0/igtr8KWsARUVSv+/CQiIiKR4xIaIhKNhPRc7D2dgod5pRgo64q542WwNDVo1TVtLQ3xul8fbD1yG3tPp2DRxMZ3ciYiIlIHLOCJSHBPCsqw70wKfkl5CitTfaydNQD9e5mr7PrD+lgh81ERfo7JggObWomISM2xgCciwZRXVuPnmEz8fCULGhIJZnj3xAR3O2hrqX5134zRvZD9iE2tRESk/rgGnojanVKpRFzSY7y/4wqOXrqLQU5dsWGpByZ5ObRJ8Q48b2pd9ltTa/DheBSwqZWIiNQUC3gialf3n5bg3/tvYMvhBOjrauJvrw7Ciqn9YGak1+b3rmlqfVZejZDDbGolIiL1xCU0RNQuysqrcPRSBk5fy4GutiZe9XHC2ME20NRo33kEW0tDvD6pD74MS8DeiGQseql3u96fiIiotVjAE1GbUiiViE54iIPn0iAvqcCoAd0w3bsXjAx0BIvJvbclMj3t8VNMJuyspRgz0EawWIiIiJqLBTwRtZnMh0XYE5GM1HuFcOxmhLdn9odjNyOhwwIATB/dE1mPi7DnVDJsuxpCZsumViIiUg9cA09EKldcVond4b/io2+v4nF+KV7z6411i4aIpngHft+p1dxID1sOxyO/iE2tRESkHljAE5HKKBRKnL2eg39si8b5mw/gM7QHPl3miVH9u0OjmbuotocuetoImOGGZxXVCDkcj8oqNrUSEZH4cQkNEalESk4B9pxKRtbjYvS2M8Grvs6wtTAUOqw/ZWthiDcm9UFIWAL2RCRjyctsaiUiInFjAU9ErVJQXI4fz6Yi+vYjmEp1sWKqK9x7W0Iiwhn3xgztbYlJXvY4EZ0JB2spxgxiUysREYkXC3giapGqagVOX8vBkUsZqK5W4JXh9pjk6QBdHU2hQ2sR/1E9kfWoGHsikmFj0QVOtiZCh0RERNQgroEnomZLyMjF+q9iceBsKnr3MMHHb3pg+uhealu8AzVNrX1hbqyHkMMJbGolIiLRYgFPRE32tKAMwYfisWn/TSiUSqyZ2R9rZg2AlamB0KGphIGeNt6a7oZnlWxqJSIi8RJ8CU1JSQmCgoIQHh4OuVwOmUyG1atXY/z48S8879q1awgNDUViYiJSU1NRVVWFpKSkBo9NSUnBv//9b1y9ehXV1dVwdXXF2rVr4e7u3hYfiajDqaisxs9XsvBTTCYkEmCGd09McLeDtlbHmwOwsTDEm5P6YMvhBOyJSMLil3qr1Xp+IiLq+AT/6xsQEIBjx45hzZo12LZtG2QyGQICAhAVFfXC82JiYhAbGwt7e3v07t34UyOys7Mxb9485OTk4OOPP0ZQUBD09fXx2muv4fr166r+OEQdilKpRFzSE7y/8wqOXMzAIKeu+HSpJyZ5OXTI4r3GEBdLvDLcHudvPsC5G/eFDoeIiKgOQWfgo6KicPnyZQQHB8PX1xcA4OnpiezsbAQGBsLb27vRc1etWoWAgAAAwIYNG5CQkNDgcdu3b0dlZSW+/vprWFpaAgBGjRqFSZMm4fPPP8e+fftU/KmIOoYHuSXYezoFtzPyYGPRBe/NG4Te9qZCh9Vupo183tS6NyIZtmxqJSIiERF0Ci0iIgJSqbTOchmJRAJ/f3+kp6cjNTW10XM1NJoW+o0bN9C3b9/a4h0AtLS0MGrUKPzyyy94/Phxyz8AUQdUVl6FA5GpWP9VLNLvyzHPxwkfvubeqYp34HlT67LJfdHVWA9b2NRKREQiImgBn5KSAplMVq8Yd3FxAQAkJye3+h4VFRXQ0dGpN14zlpKS0up7EHUESqUS0QkP8b87YhAem4Xh/azx2TJP+A7tAc0mfmHuaAz0tBEwoz/KK6uxhU2tREQkEoL+VS4oKICxsXG98ZqxgoKCVt9DJpPh119/RVlZWZ3xmvXv+fn5rb4HkbrLfFiEz/Zcx47jiTCT6uL9RUPxml8fGHWp/+W3s7Hp2gVvTuqL9PtyfH8qCUqlUuiQiIiokxP8KTQverqDKp78sGDBApw5cwZ/+9vf8Ne//hX6+vr47rvvcOvWLQBNX4pTw9xcuK3hLSykgt2bGqbuOZGXVOD78Ds4GX0X0i46eGv2QPi420FDQ72fuqLqvLxkIcXTonLsP52Mfk4W8BvuqNLrdwbq/rvSUTEv4sOciJPY8iJoAW9iYtLgLHthYSEANDg731xeXl747LPPEBgYCB8fHwDPZ+XXrFmDTZs21Vkb3xS5ucVQKNp/Bs7CQoonT4ra/b7UOHXOiUKhRNTN+zgUlYay8mqMG2yLaaMcYaCnjdzcYqHDa5W2yovvEBvcycjF9sPxMNbTgnMPNrU2lTr/rnRkzIv4MCfiJEReNDQkL5w0FnQJjUwmQ1paGhSKuutKa9a+Ozs7q+Q+/v7+uHTpEn766SecOnUKJ06cgFKphL6+PlxdXVVyDyJ1kZpTiI92XcV3J5Nga2GID19zx6u+zjDQ0xY6NFHTkPze1BoSloA8+TOhQyIiok5K0ALe19cXcrkckZGRdcbDwsLg6OgImUymsntpaWmhV69esLe3h1wux4EDBzB9+nTo6+ur7B5EYlZQXI4dxxLx6fdxKCqtxIqprnjv1UGwtRRuWZi6qdvUmoDKqmqhQyIiok5I0CU03t7e8PDwwLp161BQUABbW1uEhYUhLi4OISEhtcctXLgQsbGxdXZazcvLQ2xsLAAgKysLABAeHg4AsLGxgZubGwDg8ePH2L17NwYNGgQDAwOkp6dj586dkEqleOedd9rroxIJpqpagdPXcnD0UgaqqhWY5GWPSV720NMRvAVGLdl07YKlr/RF8KF4fHcyGa/5cadWIiJqX4L+BZdIJAgJCcGmTZsQFBQEuVwOmUyG4OBgjBs37oXnpqSkYM2aNXXGal77+/sjMDAQwPOZ98TERBw8eBDFxcWwsrKCn58fVq5cCUNDzjxSx3Y7Iw97TyfjQW4p+vcyx7zxTrAyMxA6LLU32NkCk4c74Njlu7C3lmL8EFuhQyIiok5EouQz0ZqFTaxUQ8w5eVpYhv1nUhGX/ASWJvqY6+OEgbKuQofVLtorLwqlEpsP3kJCRh7enTsQLnada6Or5hDz70pnxryID3MiTmxiJaI2VVFZjaMXM7BuxxXEp+fCf3RPfPzmsE5TvLcnDYkESye7oquJPr5kUysREbUjFvBEHYBSqcQvyU/w/s4rCLuYgQGyrtiw1BOThztAW0tT6PA6LAM9Lbw13Q0VVYrfdmplUysREbU9FvBEau5BbgmCDtzE5kPx0NXWxF/nDsSqaf1gbqwndGidQvffmlozHhRh90nu1EpERG2Pj6EgUlNl5VU4fvkuTl3Nho62BuaNd8LYwTbQ0uT38vY2yNkCU0Y44Oilu3CwNmJTKxERtSkW8ERqRqlUIibxEQ6cTUVhcQVGunXDjDG9YNxFR+jQOrUpIx2R9agY+86kwNaiC5taiYiozXCqjkiNZD0qwsY917HjWCJMDXWxbtEQvD6pD4t3EXje1NoXFib63KmViIjaFAt4IjVQXFaJ704l4f++vYr7uaVY8nJvvL94KHp1NxY6NPoDfV0tvDXDDZVVCgQfikdFJZtaiYhI9biEhkjEFAolzt+6j0NR6Sh5Volxg20xbZQjuuhpCx0aNaKbeRcsndwXm0Pj8d3JJLw+qQ93aiUiIpViAU8kUqn3CrEnIhmZD4vg3MME832d0cOSuwerg0FOFpg60hFHLmbA3loKn6E9hA6JiIg6EBbwRCJTWFyOg+fScCnhIUwMdbBsSl949LHiLK6amTzCAVmPirDvTCp6WBqyqZWIiFSGa+CJRKKqWoGTsVn4x/YYxCQ+gp+nPT5d5gnPvtYs3tWQhkSCN1/pCyuz502tuYVsaiUiItVgAU8kAol38/DPr2OxPzIVTrYm+PhND8wc0wt6OvxHMnWmr6uFgOluqKpWIPgwm1qJiEg1WMATCehpYRm2HI7Hv/bdQFW1Am/P6I+1s/rD2sxA6NBIRbqZd8HSV1yR+ZA7tRIRkWpweo9IAJVV1fj5ShZ+is4EAPiPcsRLHnbQ1tIUODJqCwOdumLaSEeE/dbU6sumViIiagUW8ETtSKlU4kbqU/xwOgVPC59hqIsF5oxzgrmxntChURt7ZYQDMh8VYf+ZVPSwMERveza1EhFRy3AJDVE7eZhXiqAfb2JzaDx0tDXx7tyBWOXvxuK9k2BTKxERqQoLeKI29qyiCj+eS8UHO68g7V4h5o6T4cPX3NHXwUzo0Kid1TS1Viu4UysREbUcl9AQtRGlUokriY9w4GwqCoorMMLNGjO9e8HYUFfo0EhAz3dqdcXmg7ewK/xXvPlKXz4mlIiImoUFPFEbyH5cjD0RyUjOLoC9tRSr/N0gszEWOiwSiYGyrpg6yhFhFzJgb22ECe5saiUioqZjAU+kQiXPKhF2PgORv+Sgi542Fr3kgtH9u0NDgzOsVNcrwx2Q+bAIByKf79Tah02tRETURFwDT6QCCoUSUTfu4R/bYhD5Sw7GDrLBp8s8MWagDYt3atAfm1q/DEvA08IyoUMiIiI1wQKeqJXS7hfik93XsCs8Cd3MDfDPJe5YMMEFhvraQodGIqevq4W3ZvRHtUKJ4EPxKGdTKxERNQELeKIWyi96hq9OJGLD7jjkF5dj2eS++Pv8wbCzkgodGqkRazMDLJvcF9mPirEr/Ffu1EpERH+Ka+CJmqmqWoHI6/dw9FIGyiuq8bKHHV4Z7gB9Xf46UcsMkHXFtNE9cfh8OhyspJgwzE7okIiISMQErThKSkoQFBSE8PBwyOVyyGQyrF69GuPHj3/hedeuXUNoaCgSExORmpqKqqoqJCUlNXhsZmYmNm/ejGvXriE/Px9WVlaYMGECli1bBiMjo7b4WNSB3bmbh72nU3DvaQkGu1hixmhHdDPvInRY1AG84mWPrIdFOHA27XlTK/cJICKiRgi6hCYgIADHjh3DmjVrsG3bNshkMgQEBCAqKuqF58XExCA2Nhb29vbo3bt3o8fl5eVhzpw5+OWXX2rvMXv2bHz33XdYuXKlqj8OdWC5hc8QEpaA/7fvBsorq/HWdDd8uNSTxTupjEQiweuT+sDa3ABfHrmNpwVsaiUiooYJNgMfFRWFy5cvIzg4GL6+vgAAT09PZGdnIzAwEN7e3o2eu2rVKgQEBAAANmzYgISEhAaPO3fuHPLz8xEUFAQvL6/aexQXF+PLL79ETk4ObG1tVfzJqCOprKpG+JUsnIjOhBLAtJGOeMnDDjramtx8h1ROX1cLb013w0e7riH4UDz+sXAIdLU1hQ6LiIhERrAZ+IiICEil0jrLZSQSCfz9/ZGeno7U1NRGz9XQaFrYWlrPv58YGhrWGZdKnzcZ6ujoNDds6iSUSiVupDzF+zuv4PCFDLj1MseGpR6YMtIROiyoqA1ZmRlg+RRXZD8uxq6f2dRKRET1CVbAp6SkQCaT1SvGXVxcAADJycmtvse4ceNgY2ODjRs3IjU1FSUlJYiJicG3336LKVOmwNLSstX3oI7nUV4p/vPjLXwRegtamhr4n7kDsdrfDV2N9YUOjTqJ/r3M4T+6J2ISH+HU1WyhwyEiIpERbAlNQUEBHBwc6o0bGxvXvt9ahoaGOHDgAN566y1MmjSpdtzf3x8bNmxo9fWpY3lWUYXjlzNx6moWtDQ1MGecDOOH2EJLk09bpfY3ycsemY+KcODs851a+7KplYiIfiPoU2hetIZYFeuL5XI5AgICUFZWhqCgIFhYWCAhIQFbtmyBQqHA559/3uxrmpsb/vlBbcTCgs8XbwtKpRIXbtzD18duI7fwGcYN7YElk/rC1EjvT89lTsSpo+Tlb4uH4d0vzmPb0UQE/cUbVmYGQofUYh0lJx0N85+7Jq0AACAASURBVCI+zIk4iS0vghXwJiYmDc6yFxYWAvh9Jr41duzYgdu3b+PcuXMwNzcHALi7u8PExAR///vfMWvWLLi7uzfrmrm5xVAo2n9NqoWFFE+eFLX7fTu67MfF2BuRjKTsAthbSbF8sitktsaoKq/EkyeVLzyXORGnjpaXlVNd8fG31/DRjmi1bWrtaDnpKJgX8WFOxEmIvGhoSF44aSzY2gCZTIa0tDQoFIo64zVr352dnVt9j8TERHTr1q22eK/Rr18/AHhhoyx1bCXPKrEnIhkffhOLnCfFWDTRBR8sHgqZbeu/OBKpkpWpAZZPfd7U+i2bWomICAIW8L6+vpDL5YiMjKwzHhYWBkdHR8hkslbfw9LSEvfv38eTJ0/qjN+4cQMAYGVl1ep7kHpRKJU4f/M+/rEtBpHXczBmkA0+W+6FMYNsoKHBx0KSOLn1NMd07564kvgIJ2PZ1EpE1NkJtoTG29sbHh4eWLduHQoKCmBra4uwsDDExcUhJCSk9riFCxciNja2zk6reXl5iI2NBQBkZWUBAMLDwwEANjY2cHNzAwDMmzcPx44dw+uvv46lS5fC0tISt27dwtatWyGTyTBy5Mj2+rgkAun35dgTkYSMB0WQ2Rpjga8z7KzEtaaNqDF+nvbIfFiEH8+looeVIVzZ1EpE1GlJlAL+e2xxcTE2bdqEkydPQi6XQyaTYfXq1fDx8ak9pqEC/sqVK1i0aFGD1/T390dgYGDt6/j4eGzZsgW3b99GYWEhunXrhrFjx2L58uUwNTVtdsxcA69+5CUVOBiVhou3HsDYUAezx8rg2deq1Y3SzIk4deS8PKuowobv4lBQVI71S9xhYaIejzbtyDlRZ8yL+DAn4iTGNfCCFvDqiAW8+qhWKBAZdw9hFzNQUVkNX/cemDzcAfq6qvmHJ+ZEnDp6Xh7nl+Kjb6/BzEgP6xYOga6O+JtaO3pO1BXzIj7MiTiJsYAX9DGSRG3l18x87DmdjHtPSuDqaIZXfZzQzbyL0GERtZrlb02t/zlwE9/8fAfLp7iq5LG7RESkPljAU4eSJ3+G/ZGpuPrrY3Q11kPAdDcMcurKAoc6lJqm1tCodNhbS/Gyh73QIRERUTtiAU8dQmWVAidjs3A8+i6USmDqSEe87GEHHTV8ZjZRU/h52iPzUTEOnkuDnaUUro5saiUi6ixYwJPau5n6FD+cTsHjgjIMdrbA3HEydFWT5j6ilpJIJHjdrzce5pZg65EEfLDEHZb8756IqFMQ7DnwRK31KL8U//nxJv6/g7egqSnB/8wZiIDpbizeqdPQ09FCwPTnj80NDr2F8opqgSMiIqL2wBl4UjvlFdU4Hn0XJ2OzoKmpgdljZfAZagstTX4fpc7H0tQAy6e4IujHm/j6pztYMZVNrUREHR0LeFIbSqUSV399jP2RqcgvKoeXqzVmje0FE0NdoUMjElS/nuaY6d0LP55Lg4O1FC97sqmViKgjYwFPaiHncTH2nk7Gr1kFsLM0xIqprnCyNRE6LCLReMnDDpmPinAwKg09rAzRz9Fc6JCIiKiNsIAnUSt9VomwCxmIvH4P+rqaWDjRBd4DukNDg0sEiP5IIpHgtZf74P7TUmw7chsfLB4KS1MDocMiIqI2wEXDJEoKpRIXbt7HP7bH4ExcDkYP7I7Plnth7CAbFu9EjdDV0UTAjN+aWg/Fs6mViKiDYgFPopN+X44Nu+Pwzc+/wsrUAOuXuGPRRBcY6msLHRqR6Fma6GPF1H6497QEX/90B0qlUuiQiIhIxbiEhkRDXlKB0Kg0XLj1AEZddPDmK33g5WrNJ2oQNZOroxlmjumFH8+mwd5aCj82tRIRdSgs4Elw1QoFIq/fQ9iFDFRUVmPisB6YMsIR+rr8z5OopV4aZofMh0UIPZcGO0tD9OvJplYioo6CFRIJKikrH3sikpHzpASuDqaY5+OM7l27CB0Wkdr7Y1Pr1iO3sX4Jm1qJiDoKroEnQeTJn2HrkQRs3PsLysqrsdrfDe/MGcjinUiFdHU08dYMN0gkwOZD8XhWUSV0SEREpAKcgad2VVmlwKmrWTh2+S6USmDKCAe87GkPXW1NoUMj6pAsTPSxYlo/bNp/A1+fuIOV0/qxr4SISM2xgKd2cyvtKfaeTsHj/DIMcuqKueOdYGGiL3RYRB2eq4MZZo2R4cDZVPwUk4lJXg5Ch0RERK3AAp7a3OP8UvxwOgU303JhZWaAd2YPYEMdUTubOKwHMh8V4VBUOuyspHDj7yARkdpiAU9tpryiGidi7iL8ShY0NTUwa2wv+A7tAS1Ntl4QtTeJRIIlL/fG/aclz3dqXTIUVmxqJSJSS6ykSOWUSiWu/voY63bG4PjlTAztbYlPl3riZQ97Fu9EAtLV1kTA9OdNrcGh8SgrZ1MrEZE6YjVFKnXvSTH+3w+/4MuwBHTR08bf5w/GssmuMJXqCh0aEeH3ptb7udyplYhIXXEJDalE6bMqHLmYgTNxOdDX1cSCCc4YM9AGGhp82gWR2PyxqfVEdCZeGe4gdEhERNQMLOCpVRRKJS7FP0DouTQUlVbCe2B3+I/uCamBjtChEdELTBzWA1mPinD4fDrsrAzRv1dXoUMiIqImErSALykpQVBQEMLDwyGXyyGTybB69WqMHz/+heddu3YNoaGhSExMRGpqKqqqqpCUlFTvuM2bNyM4OLjR6+zfvx8DBw5s9eforDIeyLEnIhnp9+XoZWOEtbMHwMHaSOiwiKgJJBIJFtc0tR5NxPrFQ2FlxqZWIiJ1IGgBHxAQgMTERLz77ruwtbXF4cOHERAQgK1bt8Lb27vR82JiYhAbGwtXV1doaWkhISGhweNmzZqFUaNG1Rtfu3Ytqqur4ebmprLP0pnISytwKCoNF24+gLSLDt6Y1Ade/ayhwc1hiNRKTVPrR7uuYfOheKxbOAT6uvyHWSIisRPs/9RRUVG4fPkygoOD4evrCwDw9PREdnY2AgMDX1jAr1q1CgEBAQCADRs2NFrAW1tbw9raus7Y7du38eDBA6xYsQKamtz9szmqFQqc++U+Dp9PR3llNXzde2DKCEcY6PEPPpG66mqijxVTXfHv33ZqXeXPnVqJiMROsKfQREREQCqV1lkuI5FI4O/vj/T0dKSmpjZ6roZGy8MODQ0FAEyfPr3F1+iMkrLy8X/fXMWeiGTYW0vx4evDMHe8E4t3og6gr4MZ5oyVIS75CY5HZwodDhER/QnBqq+UlBTIZLJ6xbiLiwsAIDk5GTKZTKX3rKiowPHjx+Hu7g57e3uVXrujyi8qx4GzqbiS+AjmRrpYNa0fhrhYcIaOqIPxde+Bu4+KEHY+HXaWhhggY1MrEZFYCVbAFxQUwMHBod64sbFx7fuqFhERgcLCQsycOVPl1+5oKqsUOHU1C8cvZ6JaocTk4Q7w87KHrjaXHRF1RBKJBItfet7Uuv0Ym1qJiMRM0PUPL5rFbYsZ3tDQUBgaGmLixIktvoa5uaEKI2oeCwtpu9zn2p1H2BEWj/tPS+Dhao03p/aDtXmXdrm3ummvnFDzMC8t9883vfCX/0Qh5Mht/OvtUTDQ01bJdZkTcWJexIc5ESex5UWwAt7ExKTBWfbCwkIAv8/Eq8qDBw8QHR2NmTNnQl9fv8XXyc0thkLR/jsXWlhI8eRJUZve43F+KfadScWN1KewMtXHX2YPgFtPc0ChaPN7q6P2yAk1H/PSOhIAy6e44t/7bmDjrqtY5d+v1U+YYk7EiXkRH+ZEnITIi4aG5IWTxoIV8DKZDKdOnYJCoaizDj45ORkA4OzsrNL7hYaGQqFQcPlMA8orq3EiOhPhV7KgqSHBzDG94Du0B7S1BOtxJiIB9bE3xexxMuw7k4ITl+9i8ghHoUMiIqI/EKyA9/X1xcGDBxEZGQkfH5/a8bCwMDg6Oqq0gVWpVOLw4cOQyWQYMGCAyq6r7pRKJeKSnmBfZAry5OXw7GuFWWNlMJXqCh0aEQnMd6gtMh/KEXYhA3ZWUja1EhGJiGAFvLe3Nzw8PLBu3ToUFBTA1tYWYWFhiIuLQ0hISO1xCxcuRGxsbJ2dVvPy8hAbGwsAyMrKAgCEh4cDAGxsbOpt0BQTE4OcnBz87W9/a+uPpTbuPS3B3ohk3MnMh62FIZa+2hcudqZCh0VEIvF7U2spth+7jQ8Wu8OaTa1ERKIgWAEvkUgQEhKCTZs2ISgoCHK5HDKZDMHBwRg3btwLz01JScGaNWvqjNW89vf3R2BgYJ33Dh06BG1tbUydOlW1H0INlT6rwtFLGTgTlwNdbU3M93XGmEHdodmKZ+sTUcek89tOrf/37VVsDr2F9xcN5U6tREQiIFEqle3fkanG1LWJVaFUIjrhIX48l4aikgqMGtAd0717wshAR4VRdi5sNhIn5kX1fs3Mx7/23cAAmTlWT3drdlMrcyJOzIv4MCfixCZWEsTdh3LsOZWMtPty9OxuhDUz+8Oxm5HQYRGRmuhtb4o542T44UwKjl++iylsaiUiEhQL+A6sqLQCh86n4/yN+5AaaON1vz4Y7mbd6kfCEVHn4zPUFpmPinDkQgbsLKUY6MSmViIiobCA74CqFQqc++U+wi6ko6y8Gr7uPTBlhCMM9JhuImoZiUSCRRNdcO9pCXYcv433Fw1FN27wRkQkCHYudjDJ2QX46Ntr2BORDDsrKf7vdXfMHe/E4p2IWk1HWxMB/m7Q0tRA8KF4lJVXCR0SEVGnxAK+g8gvKsf2o7cRuOc6Sp5VYtW0fnh37kDYWDTeAEFE1FzmxnpYObUfHuWVYcexRCj4HAQionbHaVk1V1WtQMTVbBy9fBfV1Uq8MtwBkzztoaujKXRoRNRB9bY3xZzxMvxwOgXHLt3F1JFsaiUiak8s4NVYfHou9p5OwaO8UgyUdcXc8TJYmnKjFSJqez5DbJH5sAhHLmbAzsoQg5wshA6JiKjTYAGvhh4XlGH/mRT8kvIUVqb6WDtrAPr3Mhc6LCLqROo0tR5LxAeL2dRKRNReWMCLXPTthzgUlYY8eTlMpbqwt5YiPj0PmhoSzPDuiQnudtDWYisDEbU/HW1NvFW7U2s83l80lA3zRETtgJWfiEXffohdP/+KXHk5lADyisrxS8pT2Fl1wYalHpjk5cDinYgEZWakh1XT+uFJQRl2HmdTKxFRe2D1J2KHotJQUaWoN15YXAEzIz0BIiIiqs/FzhRzxzvhRupTHL2YIXQ4REQdHgt4EcuVlzdrnIhIKOMG22CEmzWOXrqLX5KfCB0OEVGHxgJexMyNdJs1TkQklJqmVsduUuw4nogHuSVCh0RE1GGxgBex6d69oPNfa9x1tDQw3buXQBERETVOW0sTq/3doKOlgS9C41H6jDu1EhG1BRbwIublao3FL/eGuZEuJHg+87745d7wcrUWOjQiogaZGelhlb8bnhaUYcex22xqJSJqA3zel8h5uVrDy9UaFhZSPHlSJHQ4RER/yrmHCeaOd8KeiGQcvZiBpdMHCB0SEVGHwgKeiIhUbtxgG2Q+LMLRS3fRz8kSMmtDoUMiIuowuISGiIhUTiKRYOFEZzh2M0LQD3G4/5RNrUREqsICnoiI2sTzptZ+0NXWwuZDbGolIlIVFvBERNRmzIz08PfF7mxqJSJSIRbwRETUplx7mmOejxNupuXiyAXu1EpE1Fos4ImIqM2NHWSDkf274djlu4hL4k6tREStwQKeiIjanEQiwcIJz5tad55IxD02tRIRtVizC/jMzEycP3++ztjNmzexYsUKzJ07F/v371dZcERE1HFoa2kiYLobdLU1ERx6C6XPKoUOiYhILTW7gP/Xv/6FHTt21L7Oy8vD0qVLcfHiRaSkpODDDz/E6dOnm3y9kpISfPLJJxg5ciT69++P6dOn48yZM3963rVr1/CPf/wDU6dOhaurK1xcXF54/J07d7B69Wp4eHjAzc0Nvr6+CA4ObnKcRETUeqZSXaya1g9PC59h+7FEKBRsaiUiaq5mF/AJCQkYPnx47esTJ06guLgYhw4dQnR0NAYMGIBdu3Y1+XoBAQE4duwY1qxZg23btkEmkyEgIABRUVEvPC8mJgaxsbGwt7dH7969X3jspUuXMHv2bOjo6ODTTz/Fjh07sGzZsibHSEREquPcwwSv+jjhVlouwi6mCx0OEZHaafZOrHl5ebC0tKx9feHCBQwePBjOzs4AAD8/P2zdurVJ14qKisLly5cRHBwMX19fAICnpyeys7MRGBgIb2/vRs9dtWoVAgICAAAbNmxAQkJCg8eVlpbivffew4wZM/Dhhx/Wjnt6ejYpRiIiUr0xg2xw92ERjl/OhL2VFENcLP/8JCIiAtCCGXh9fX0UFRUBAKqrqxEXF4ehQ4fWvq+np4fi4uImXSsiIgJSqRTjx4+vHZNIJPD390d6ejpSU1MbD1yjaaGHh4fj6dOnWLp0aZOOJyKitieRSLBgggt6djfCzuN3cO9J0/5uEBFRCwp4JycnHDlyBPn5+Thw4ABKS0sxYsSI2vfv3bsHMzOzJl0rJSUFMpmsXjFes549OTm5ueHVc/XqVZiYmCAjIwNTp05F37594eXlhfXr1zf5iwYREametpYGVvu7QU9HE5sPxaOETa1ERE3S7AL+jTfeQFJSEoYPH46PPvoIffr0qTMDf+nSJfTt27dJ1yooKICxsXG98ZqxgoKC5oZXz+PHj1FWVoa3334bfn5++Oabb7B06VIcPXoUS5cuhZK7AhIRCcZUqotV/v2QW/gM24+yqZWIqCmavQZ+zJgx2LVrF86cOQNDQ0MsWLAAEokEAJCfnw9ra2tMmzatyderObe57zWVUqlEeXk5AgICahtXPTw8oKuri48++gjR0dF1mnL/jLm5YatjaikLC6lg96aGMSfixLyIz4tyYmEhxfJn1Qg5eBMn43KwyK9pk0DUevxdER/mRJzElpdmF/AA4O7uDnd393rjpqamzXo0o4mJSYOz7IWFhQDQ4Ox8c5mYmAAARo4cWWd89OjRAIDbt283q4DPzS0WZIbIwkKKJ0+K2v2+1DjmRJyYF/FpSk6GyswxekB3/HgmBRZSXQztzabWtsbfFfFhTsRJiLxoaEheOGmskp1Yq6qqcPLkSRw4cABPnjR9i2yZTIa0tDQoFIo64zVr32uebNMajV2jZulMU5thiYiobc33dUYvGyN8deIOctjUSkTUqGZXr59//jlmzJhR+1qpVOK1117D2rVrsX79ekyePBlZWVlNupavry/kcjkiIyPrjIeFhcHR0REymay54TV4D4lEUu+58jWvBwwY0Op7EBFR62lraWDVtOdNrcGhbGolImpMs5fQXLhwoc6Sk8jISFy9ehVvvvkm+vTpg48//hjbt2/HJ5988qfX8vb2hoeHB9atW4eCggLY2toiLCwMcXFxCAkJqT1u4cKFiI2NRVJSUu1YXl4eYmNjAaD2C0N4eDgAwMbGBm5ubgCAXr16Yc6cOQgJCYFCocDgwYORkJCA4OBgjB49uk4DLhERCctUqovV/m7YuPc6th29jbUzB0BDo/X9UEREHUmzC/iHDx/C3t6+9vXZs2dha2uLd999F8DzR0MeO3asSdeSSCQICQnBpk2bEBQUBLlcDplMhuDgYIwbN+6F56akpGDNmjV1xmpe+/v7IzAwsHb8gw8+gLW1NQ4ePIgvv/wS5ubmmD9/fr3ziYhIeDJbY8yf4Izd4Uk4fCEdM7x7CR0SEZGoNLuAr6yshKamZu3rK1eu1JmR79GjR7PWwRsaGmL9+vVYv359o8d899139cY8PDzqzMi/iJaWFlauXImVK1c2OS4iIhLOmIE2yHxYhBPRz3dqZVMrEdHvmr0G3traGjdu3ADwfBY8Ozu7zhNpcnNzYWBgoLoIiYioU3rVh02tREQNaXYBP2nSJISFhWH58uVYvnw5DA0N4e3tXfv+nTt3YGdnp9IgiYio86ndqVWXTa1ERH/U7AJ++fLl8Pf3x40bNyCRSLBx40YYGRkBAIqKihAZGQkvLy+VB0pERJ2PieHzptZc+TNsO3KbO7USEaEFa+B1dHTw6aefNvhely5dcPHiRejp6bU6MCIiIgCQ2RhjwQRn7ApPwqHz6Zg5hk2tRNS5tWgn1sZoaGhAKhXXVrNERKT+vH9rav0pJhN2VoYY1sdK6JCIiATTogK+tLQUO3fuREREBHJycgAAtra2mDBhAt544w02sRIRkcq96uuMnCcl+PqnO+hm3gU9LBvfZpyIqCNr9hr4goICzJo1CyEhIXj69Cn69OmDPn36IDc3F1u2bMGsWbNQUFDQFrESEVEnpqWpgVX+/WCgq4XNobdQXMamViLqnJpdwH/xxRdIT0/HBx98gIsXL2Lv3r3Yu3cvLly4gPXr1yMjIwPBwcFtESsREXVyNU2tBcXl2HaUTa1E1Dk1u4CPjIzErFmzMH/+/DobOmlqauLVV1/FjBkzcPr0aZUGSUREVKOXjTEWTHDB7Yw8hEalCR0OEVG7a3YBX7NspjF9+/bF06dPWxUUERHRi4we0B1jBtng5ytZiL3zSOhwiIjaVbML+K5du+LOnTuNvn/nzh107dq1VUERERH9mVd9nCCzNcbXP91B1qMiocMhImo3zS7gx44di4MHD2Lfvn1QKBS14wqFAvv370doaCjGjRun0iCJiIj+m5amBlZPe97UGnwonk2tRNRpSJRKZbM6gPLz8zF37lxkZWXBzMwMjo6OAICMjAzk5eXBzs4O+/btg6mpaZsELLTc3GJBmqYsLKR48oQzTGLCnIgT8yI+bZ2TtPuF2LjnOlx6mGDt7AHQ1Gj23FSnxN8V8WFOxEmIvGhoSGBu3vijcpv9fzlTU1OEhoZi2bJlMDExQXx8POLj42Fqaoply5YhNDS0wxbvREQkPr26G2PhBBfcvpuP0Kh0ocMhImpzLdrIydDQEH/5y1/wl7/8pd57+/btw+7du/HTTz+1OjgiIqKmGDWgO+4+KkL4lSzYW0nh0Zc7tRJRx6Xyf2fMz89HRkaGqi9LRET0QvPGO8HJ1hjfsKmViDo4LhQkIqIO4flOrW7ooq/NplYi6tBYwBMRUYdh3EXnt51aK7D1SAKq//C0NCKijoIFPBERdSg9uxth4URnJN7NR+g5NrUSUcfToiZWIiIiMRvVvzsyHxYhPDYLdtaG8OxrLXRIREQq06QC/ptvvmnyBa9fv97iYIiIiFRl7ngn5Dwuxrc//Yru5l1gZyUVOiQiIpVoUgG/cePGZl1UIpG0KBgiIiJV0dLUwEp/N3z07VUEH4rH+iXuMNTXFjosIqJWa1IBv3v37raOg4iISOWMu+ggYLobPvv+Or4MS8A7c7hTKxGpvyYV8MOGDWvrOIiIiNqEYzcjLJrogq9/uoOD59IwZ5yT0CEREbWKoE2sJSUlCAoKQnh4OORyOWQyGVavXo3x48e/8Lxr164hNDQUiYmJSE1NRVVVFZKSkuodl5OT0+i1duzYgdGjR6vkcxARkbiN7N8NmQ+LcDI2G/ZWUni6sqmViNSXoAV8QEAAEhMT8e6778LW1haHDx9GQEAAtm7dCm9v70bPi4mJQWxsLFxdXaGlpYWEhIQX3mfx4sXw8/OrM9arVy+VfAYiIlIPc8bLkP2kGN/8/Cu6mXeBvTWbWolIPQlWwEdFReHy5csIDg6Gr68vAMDT0xPZ2dkIDAx8YQG/atUqBAQEAAA2bNjwpwV89+7dMXDgQNUFT0REakdLUwOrpvXD/9U2tQ6F1EBH6LCIiJpNsE6eiIgISKXSOktcJBIJ/P39kZ6ejtTU1EbP1WADEhERtYDRb02thSUV2HrkNndqJSK1JFglnJKSAplMVq8Yd3FxAQAkJyer7F5bt25Fv379MHDgQCxcuBDR0dEquzYREakXx25GWPySC+5k5uPHs2lCh0NE1GyCLaEpKCiAg4NDvXFjY+Pa91tLR0cHs2fPxogRI9C1a1fk5OTgm2++wWuvvYbNmzfXLt0hIqLOZYRbN9x9WIRTV583tXr1Y1MrEakPQZtYX7Thkyo2g7K0tMTHH39c+3ro0KGYOHEipk2bhs8//7xFBby5uWGr42opCws2XIkNcyJOzIv4iDEnAXMG4VFBGXaF/4q+ThaQ2ZoIHVK7E2NeOjvmRJzElhfBCngTE5MGZ9kLCwsB/D4Tr2r6+vqYOHEitm3bhry8PJiZmTXr/NzcYigUyjaJ7UUsLKR48qSo3e9LjWNOxIl5ER8x5+RNvz74aNdVfPJVDD5Y4g6jTtTUKua8dFbMiTgJkRcNDckLJ40FWwMvk8mQlpYGxX81ENWsfXd2dm6ze9fcUxWz/EREpL5qmlrlpZXYGpbAplYiUguCFfC+vr6Qy+WIjIysMx4WFgZHR0fIZLI2uW9ZWRlOnToFe3t7mJqatsk9iIhIfThYP29q/TWrAAci2dRKROIn2BIab29veHh4YN26dSgoKICtrS3CwsIQFxeHkJCQ2uMWLlyI2NjYOjut5uXlITY2FgCQlZUFAAgPDwcA2NjYwM3NDQAQGBgIhUKBQYMGwczMDPfu3cO3336L7OxsbNmypb0+KhERidzwfs+bWiOuZcPe2hDD+3UTOiQiokYJVsBLJBKEhIRg06ZNCAoKglwuh0wmQ3BwMMaNG/fCc1NSUrBmzZo6YzWv/f39ERgYCOD5Mp39+/cjLCwMJSUlMDQ0xKBBg/DPf/4TQ4YMaZsPRkREamn2WBlyHhdjV3gSbLoacqdWIhItiVKpbP+OTDXGJlaqwZyIE/MiPuqUE3lpBT7+9ioAdPimVnXKS2fBnIgTm1iJiIhEzMhABwHT+9c2tVZVs6mViMSHBTwREdEf2FtLseSl3s+bWs+mCh0OEVE9gm7kREREJEZe/axrm1odrKVsaiUiUeEMPBERUQNmj+uF3nYm2BWehLsP5UKHQ0RUiwU8ERFRAzQ1NLBikKDrOAAAIABJREFUWj8YGWgj+FA85CUVQodERASABTwREVGjappai0or8SWbWolIJFjAExERvYC9tRRLXu6NpOwC7I9kUysRCY9NrERERH/Cy9UamQ+LcOpqNuytpBjZn02tRCQczsATERE1wayxvdDH3hS7TyYh4wGbWolIOCzgiYiImkBTQwMrprrCuIsOgg/Fo5BNrUQkEBbwRERETSQ10EHAdDeUlLGplYiEwwKeiIioGWqaWpOzC7D/DJtaiaj9sYmViIiomTxdn+/UeupqNuyt2dRKRO2LM/BEREQtwKZWIhIKC3giIqIWqGlqNTFkUysRtS8W8ERERC1Up6n1cDybWomoXbCAJyIiagU7Kyle8+uD5JxC7DuTInQ4RNQJsImViIiolTz6WiHzYRHCY7Ngby3FqP7dhQ6JiDowzsATERGpwIwxPdHXwRTfnUxC+n02tRJR22EBT0REpALPm1r7wcRQF1sOx6OwuFzokIiog2IBT0REpCKG+tq1Ta0h3KmViNoIC3giIiIVqmlqTckpxA9saiWiNsAmViIiIhX7/9u797Co6vwP4O8zDHdmuI7cxcEJUEQxdQlv5IWefmYplo+tSrVq/kox2ja7rNE+v1YfdW1lUyJ13XrSzNo1IaM08BKlqOR1NVNASFTwAjiMIiow8/sDZmTkIgLDOQPv1/P01Hzne875Hj+ezofD93s+Uf29ce7ydew4WIw+3gqMGsRFrUTUeUR9Al9VVYXFixdj5MiRGDhwIKZMmYJdu3bdd7tDhw7h7bffxqRJkxAeHo7Q0NA2HW/r1q0IDQ3F0KFDOzp0IiKiVj0T0xfhfdyxMfMMzpZUij0cIupGRE3gExIS8M033yAxMRFr166FRqNBQkICsrOzW93uwIEDyM3NRVBQEMLCwtp0rLKyMixfvhwqlaozhk5ERNQqmUzA/xoXtW7lolYi6jyiJfDZ2dnIycnB4sWLMXXqVERHR2P58uWIjIzEsmXLWt123rx52LVrF1atWoWHH364Tcd77733MHjwYIwcObIzhk9ERHRfLo62WPD0QNy8XYsPuaiViDqJaAl8VlYWFAoFxo0bZ2oTBAFxcXEoLCxEQUFBi9vKZA827MzMTPz0009499132z1eIiKi9gjs5YJZE/qh4EIlPt/JRa1E1HGiJfD5+fnQaDRNknHjfPa8vLxOOU5lZSXee+89JCYmws+Pi4iIiKjr/a6fN/4nqjd+OHoRPx4vEXs4RGTlREvgtVotXF1dm7Qb27RabaccZ9myZfD29kZ8fHyn7I+IiKg9no7pi3C1Bz7LPIOzF7molYjaT9TXSAqC0K7v2mrfvn3Ytm0btmzZAhsbmw7vDwA8PV06ZT/toVIpRDs2NY8xkSbGRXoYk3qLZkXhtX9k46OvTyL5j4/CQ+kg6ngYF+lhTKRJanERLYF3c3Nr9il7ZWX9U4nmns4/iJqaGiQlJWHq1Knw9/eHTqcztRsMBuh0Otja2sLR0fGB9ltefgN6vaFDY2sPlUqBq1evd/lxqWWMiTQxLtLDmJibN2kAFm88hL+uP4A3pg+G3EacX4YzLtLDmEiTGHGRyYRWHxqLNoVGo9Hg7Nmz0OvNV+Qb576HhIR0aP/V1dW4ePEiNm/ejGHDhpn+ycjIwI0bNzBs2DD8+c9/7tAxiIiIHlSAcVHrxUp8ntU5672IqGcR7Ql8bGwstmzZgt27d2P8+PGm9vT0dKjVamg0mg7t38nJCRs2bGjSvm7dOhw5cgRr1qyBp6dnh45BRETUHr/rV1+pdfuBYgT5KBAT6S/2kIjIioiWwMfExCAqKgqLFi2CVqtFQEAA0tPTcfjwYaSmppr6xcfHIzc3F2fOnDG1VVRUIDc3FwBQXFwMANixYwcAwN/fHxEREZDL5YiKimpy3LS0NNjY2DT7HRERUVd5enRfnL98A59l5sFf5QKNf8emjhJRzyFaAi8IAlJTU7Fy5UokJydDp9NBo9EgJSUFY8eObXXb/Px8JCYmmrUZP8fFxd23EBQREZHYZDIBc58Kx+JPD+HDtBN49/lhcFfYiz0sIrICgsFg6PoVmVaMi1jJiDGRJsZFehiT1l24egNLNhxGQC9nvPH7h2Er75rlaYyL9DAm0sRFrERERGQmQOWC2U/0w9mLOny+k4taiej+mMATERGJbGhYLzwRHYTsYyX44dhFsYdDRBLHBJ6IiEgC4kYFIyLYE5sy81BwgZVaiahlTOCJiIgkoH5Ra394Kh3wYdoJXLt+W+whEZFEMYEnIiKSCGcHWyQ8HYFbd+qQmnYCNbX6+29ERD0OE3giIiIJMS1qLdFhEyu1ElEzmMATERFJjHFR64/HS/DDUS5qJSJzTOCJiIgkyLSoNSsP+Re0Yg+HiCSECTwREZEEyWQC/vep/vB0dUBq2kkuaiUiEybwREREEuXkYIsFUyJwq6YOH3JRKxE1YAJPREQkYf4qF8x5oh8KS3T4LPMMDAaD2EMiIpExgSciIpK4IaG9MHF4EH76bykXtRIRE3giIiJrMHlkMAb29cTnO/ORd56LWol6MibwREREVkAmEzD3yf7wcnVAavpJVOhuiT0kIhIJE3giIiIr4eRgi4SnB+J2TR0+TDuJmto6sYdERCJgAk9ERGRF/L2cMeeJ/igq1WFjZh4XtRL1QEzgiYiIrMyQUBUmDu+Dvf8txR4uaiXqcZjAExERWaHJo9QY2NcTm7molajHYQJPRERkhWRCw6JWN0ekpp3golaiHoQJPBERkZUyVmq9U6tvqNTKRa1EPQETeCIiIivm5+WMFyf2R1HpdWz8notaiXoCJvBERERWbnCICk+N6IO9J0qx+wgXtRJ1d0zgiYiIuoGnRqoRqfHCF7vycab4mtjDISILYgJPRETUDcgEAXMm9ofKzREfsVIrUbcmagJfVVWFxYsXY+TIkRg4cCCmTJmCXbt23Xe7Q4cO4e2338akSZMQHh6O0NDQZvsVFRXhpZdewpgxYzBw4EAMGzYM06ZNw7Zt2zr7VIiIiETn5CDHgqe5qJWouxM1gU9ISMA333yDxMRErF27FhqNBgkJCcjOzm51uwMHDiA3NxdBQUEICwtrsd+NGzfg5eWFxMRErFu3DitWrEBAQAAWLlyIdevWdfbpEBERic7X0xkvPlm/qHXD92e4qJWoGxIMIl3Z2dnZmDt3LlJSUhAbGwsAMBgMmD59OrRaLbZv397itnq9HjJZ/c8eS5YswYYNG3DmzJk2H3vatGkoKytr09P+e5WX34Be3/V/ZCqVAlevXu/y41LLGBNpYlykhzERR/pPhdi27zfMiA3BuCEBTb5nXKSHMZEmMeIikwnw9HRp+fsuHIuZrKwsKBQKjBs3ztQmCALi4uJQWFiIgoKCFrc1Ju/t5ebmBrlc3qF9EBERSRkXtRJ1X6Il8Pn5+dBoNE2SceN89ry8vE47ll6vR21tLSoqKrBp0ybs3bsXc+bM6bT9ExERSY1MEPDik/WLWlO5qJWoWxEtgddqtXB1dW3SbmzTarWddqwVK1YgPDwc0dHRWLp0KRYtWoSpU6d22v6JiIikyNG+flFrTa0eq7eewJ0aLmol6g5EnUciCEK7vntQzz//PCZMmICKigpkZ2fjr3/9K6qrqzF79uwH3ldr85EsTaVSiHZsah5jIk2Mi/QwJuJRqRR4fcYQLP4kF//OLsSrzw423WMZF+lhTKRJanERLYF3c3Nr9il7ZWUlADT7dL69fHx84OPjAwCIiYmBIAhYuXIl4uLi4OHh8UD74iJWMmJMpIlxkR7GRHzB3i6YNFKNr/cWobr6DvLOa1Ghuw0PpT2mxPRFdLiP2EMk8FqRKi5ibUSj0eDs2bPQ6/Vm7ca57yEhIRY7dkREBGpra3H+/HmLHYOIiEhKnhzRB0HeLtj/y2WU627DAKBcdxufbj+N/b9cEnt4RPQAREvgY2NjodPpsHv3brP29PR0qNVqaDQaix374MGDkMlkCAho+lotIiKi7kgmCNDdrGnSfqdWj63ZZ0UYERG1l2hTaGJiYhAVFYVFixZBq9UiICAA6enpOHz4MFJTU0394uPjkZuba/ae94qKCuTm5gIAiouLAQA7duwAAPj7+yMiIgIAsHTpUtTV1eHhhx+Gl5cXrl27hszMTGRkZGDWrFnw9PTsqtMlIiIS3bXrt5ttL9fdxvaD5xDsq0SQjwIOdnzVMpGUiXaFCoKA1NRUrFy5EsnJydDpdNBoNEhJScHYsWNb3TY/Px+JiYlmbcbPcXFxWLZsGQBg0KBB2Lx5M7799lvodDo4OjoiNDQUy5cvx6RJkyxzYkRERBLlqbRHua5pEi8TgP/sqX8KLwiAv5czgv2UUPsqEeznCj8vJ9h0sAYLEXUe0SqxWisuYiUjxkSaGBfpYUykY/8vl/Dp9tO4U3t3/ZmdXIbn/ycM4WoP/FaqQ2GJDoWlOhSV6FB1qxYAYG9rgyAfBYJ9labE3kNp36lvjCNeK1IlxUWs/B0ZERFRD2F828zW7LPNvoVmYF8vDOzrBQAwGAy4oq1GYUl9Ml9YqsPOw+dRm1v/EMvV2a7hCb0Saj8l1D5KODkwrSDqCrzSiIiIepDocB9Eh/vc96miIAjwdneCt7uTKcGvrdPj/JUb9Ul9w9P6YwVlpm18PZ0Q7Fuf0Af7KRGgcoHchlNviDobE3giIiJqE7mNDGrf+ik0Rjdv1aCo9Lpp2s2JwnLsO3nJ1D/Ix8X0pD7YVwmVmyOn3hB1EBN4IiIiajcnB1uEqz0Qrq4vjGgwGFCuu1Wf1JdUoqhEhx+Pl2DnoQsAABdH24YfAhQI9nOF2lcBhZOdmKdAZHWYwBMREVGnEQQBXq6O8HJ1xLCwXgCAOr0eF69WmZ7SF5bqcLKwHMZXQvRyc6yfdtMw/SbI2wW2chvxToJI4pjAExERkUXZyGTo7a1Ab28FHo30BwBU367FuUvXTXPp885rcfDU5Yb+AgJ6uZim3QT7KeHt4QQZp94QAWACT0RERCJwtJcjLMgdYUHuprZr12+bEvqiUh32n7yEPUcumvqrfRVm8+ldXezFGj6RqJjAExERkSS4K+zhrlDh4RAVAECvN6C04mb9XPqGOfXbDxRD31DCxlNpbyo2pfZVoI+PEvZ2nHpD3R8TeCIiIpIkmUyAv5cz/L2cMWpgfdudmjoUX76BwpJKFDY8rT905ioAYxXZhqk3DU/p/bycIZNx6g11L0zgiYiIyGrY2dpAE+AKTYCrqU138w6KGr2b/vCZK/jxeAmA+iqyfXwUpkWywX5KuCtYRZasGxN4IiIismpKJzsM0nhhkKZRFdlr1aYn9EWlOuw8dB61dXeryAb7KU3z6fuwiixZGf5tJSIiom5FEAR4ezjB2+NuFdmaWj0uXK2vIlvY8CrLo/n1VWQFAD4NVWSD/epfZckqsiRlTOCJiIio27OV360iO25IfVvVrRoUNbybvqj0ulkVWVu5DL29XRDs6wq1X33RKZWrA6fekCQwgSciIqIeydnBFgPUnhig9gRwt4psYaP59NnHLiLrkB5AfRXZxlNv1L5KuDjainkK1EMxgSciIiKCeRXZ3/XzBmBeRdaY2J8426iKrLujqYJssK8SvVlFlroAE3giIiKiFrRWRbawYfrNmfNaHGhURTawoYqsmlVkyUKYwBMRERE9gJaqyN6delOJfScvYfc9VWTvJvWucHW2E2v41A0wgSciIiLqIHeFPYaEqjAktFEV2fIq01P6wlIdvtt/TxVZP1fTm2+CvBViDp+sDBN4IiIiok4mkwnwV7nAX+WCUQP9AAC3a+pQfPm62SLZQ6ev1PcXBAT5KhCocmEVWbovJvBEREREXcDe1gYPBbjhoQA3U5uu6o4pmb9YfrPZKrKN59OziiwBTOCJiIiIRKN0vltFVqVS4MoVXX0V2YZpN4UlOmQ1riLrYmeadhPsq0QfXyUc7ZnO9TSMOBEREZFEmFWRHXC3iuz5KzdMC2QLS6+bVZH19XJuWCRbP6feX+XMKrLdHBN4IiIiIgmzlcvqn7j7KTFuSAAA4EZ1DX671PBu+hId/nu2HPtO3K0iG+R9d+qN2k/JKrLdDBN4IiIiIivj4thMFdnKW6ZpN4WlOuw5ehGZP5839TdOu1GziqzVEzWBr6qqQnJyMnbs2AGdTgeNRoP58+dj3LhxrW536NAhfPXVVzh16hQKCgpQW1uLM2fONOl34sQJbNmyBYcPH8bFixfh5OSEsLAwvPzyyxg6dKilTouIiIioSwmCAC83R3i53a0iW1tXX0W2qLUqso0WyPbupYCtnFNvrIGoCXxCQgJOnTqF119/HQEBAUhLS0NCQgLWrFmDmJiYFrc7cOAAcnNzER4eDrlcjpMnTzbb77vvvsPJkyfxzDPPICwsDDdu3MCmTZswc+ZMrFq1Co899pilTo2IiIhIVHIbGYJ8FAjyUeDRwXeryP526ToKSypRVHodp89dw4Ff7laR7e3tYkro1b6sIitVgsFgMNy/W+fLzs7G3LlzkZKSgtjYWAD1v/6ZPn06tFottm/f3uK2er0eMln9T4hLlizBhg0bmn0CX15eDk9PT7O2mpoaTJw4Ec7Ozti6desDj7u8/Ab0+q7/I1OpFLh69XqXH5daxphIE+MiPYyJNDEu0iNWTIxVZAtLK1FUokPRpeu4facOAODUUEW2cdEpZQ+rIitGXGQyAZ6eLi1+L9oT+KysLCgUCrPpMoIgIC4uDklJSSgoKIBGo2l2W2Pyfj/3Ju8AYGtri7CwMPz888/tGzgRERFRN9JiFdlGBae+23+uURVZB7OpN0E+Ctjb2oh5Cj2OaAl8fn4+NBpNk2Q8NDQUAJCXl9diAt8Rd+7cwdGjR/HQQw91+r6JiIiIrJ1ZFdlBd6vInrt03Ww+/c+NqsgGqJyhbrRI1s+TVWQtSbQEXqvVok+fPk3aXV1dTd9bwooVK3DlyhUsX77cIvsnIiIi6m7sbW0QEuiGkMC7VWQrG1WRLSrV4edfryD7WEMVWTsbqH0UZvPpPZQOYg2/2xF1EWtr7yO1xLtKN27ciA0bNmDBggWIjo5u1z5am49kaSqVQrRjU/MYE2liXKSHMZEmxkV6rCkmKhWg6XN3urJx6s2Zc9eQX3wNeeevmVWR9VA6IKS3G0J6uyOktzseCnSDk4N1vMpSanERLYF3c3Nr9il7ZWUlgLtP4jvLl19+iSVLluCFF15AQkJCu/fDRaxkxJhIE+MiPYyJNDEu0tMdYmIHICLIDRFBbgDUpiqy9RVkdSi6WIkDJ+sLThmryBqn3Ui1iiwXsTai0WiQmZlp9kYZoH7uOwCEhIR02rH+85//4C9/+QumT5+Ot99+u9P2S0REREQta1xF1uhGdQ1+a1Rw6lhBGfaeKAUA2Mll6O2jML3xRu2rhBeryDYhWgIfGxuLLVu2YPfu3Rg/frypPT09HWq1utMWsH711VdISkrC1KlTkZSU1Cn7JCIiIqL2cXG0xYBgTwwIvltFtqzylmk+/b1VZBVOtqa59Man9c5WMvXGUkRL4GNiYhAVFYVFixZBq9UiICAA6enpOHz4MFJTU0394uPjkZuba/ae94qKCuTm5gIAiouLAQA7duwAAPj7+yMiIgIAsH37drzzzjsYMGAApkyZguPHj5uNITIy0qLnSEREREStEwQBKjdHqJqpIltYqkNRQ1LfuIqst1kVWVcE9nLpUVVkRUvgBUFAamoqVq5cieTkZOh0Omg0GqSkpGDs2LGtbpufn4/ExESzNuPnuLg4LFu2DEB9sSi9Xo8TJ07g2WefbbKf5oo/EREREZG4GleRHdO4imxpfTJfWKLDqXPXsL+hiqzcRkBgr0ZTb/yU6OXu2G2ryIpWidVacRErGTEm0sS4SA9jIk2Mi/QwJg+uQnfL7FWWTarINio4FezbviqyXMRKRERERNRJPJQO8FA6YEhoLwD1r7IsKa8yTbspuqeKrJerg9m76VurIrv/l0vYmn0WFbrb8FDaY0pMX0SH+3TZubWGCTwRERERdQsymYAAlQsCGleRvVOHc5evm57SF5Y0rSIb3OhJva+nMw7+ehmfbj+NO7V6AEC57jY+3X4aACSRxDOBJyIiIqJuy96uhSqypqf0lTj46xX80FBF1sHOBrV1elMBKqM7tXpszT7LBJ6IiIiIqKu5Otsh8iEvRD7kBQDQGwy4XHHT9IR+95GLzW5XrrvdlcNsERN4IiIiIurRZIIAX09n+Ho6Y/gAXxwvKGs2WfdU2oswuqZ6zgsziYiIiIjaYEpMX9jd8155O7kMU2L6ijQic3wCT0RERETUiHGeO99CQ0RERERkJaLDfRAd7iPJ9/NzCg0RERERkRVhAk9EREREZEWYwBMRERERWREm8EREREREVoQJPBERERGRFWECT0RERERkRZjAExERERFZESbwRERERERWhAk8EREREZEVYSXWBySTCT3y2NQ8xkSaGBfpYUykiXGRHsZEmro6Lvc7nmAwGAxdNBYiIiIiIuogTqEhIiIiIrIiTOCJiIiIiKwIE3giIiIiIivCBJ6IiIiIyIowgSciIiIisiJM4ImIiIiIrAgTeCIiIiIiK8IEnoiIiIjIijCBJyIiIiKyInKxB9CTVVVVITk5GTt27IBOp4NGo8H8+fMxbty4+25bXFyMZcuW4eDBg9Dr9Rg6dCjefPNNaDSaLhh599XemKxevRopKSlN2r28vLBv3z5LDbdHuHTpEtavX49ffvkFp0+fxs2bN7FhwwZERUW1afuTJ09ixYoVOH78OGxtbTFy5Ei89dZb8Pb2tvDIu7eOxOWtt95CWlpak/ZBgwbh3//+tyWG2yPs378fX3/9NY4ePYpLly7B1dUVAwcOxIIFCxAaGnrf7Xlf6XwdiQnvK5Zz5MgRfPjhh8jLy4NWq4WzszNCQkIwe/ZsxMTE3Hd7KVwrTOBFlJCQgFOnTuH1119HQEAA0tLSkJCQgDVr1rT6F6i8vBzTp0+Hp6cnli9fDhsbG3z00UeYOXMm0tPT4ePj04Vn0b20NyZGn3zyCZycnEyfbW1tLTncHuHcuXP49ttv0b9/fzzyyCPYvXt3m7c9e/Ys4uPjERERgQ8++ADV1dVITk5GfHw80tLS4OzsbMGRd28diQsAODk54ZNPPjFrYzw6ZvPmzdBqtXjhhRfQt29flJWVYf369XjmmWewceNGREZGtrgt7yuW0ZGYGPG+0vl0Oh3UajWmTJkCLy8v6HQ6fPnll5g7dy5WrlyJJ554osVtJXOtGEgUP/zwgyEkJMSQmZlpatPr9YZnn33W8Pjjj7e67fLlyw0RERGGS5cumdoqKioMgwcPNrz77rsWG3N315GYrFq1yhASEmKorKy09DB7nLq6OtN/Z2VlGUJCQgwHDhxo07avvPKKYcSIEYaqqipTW0FBgSEsLMywdu3aTh9rT9KRuLz55puGIUOGWGpoPVZZWVmTtsrKSsPQoUMNCQkJrW7L+4pldCQmvK90rZqaGsPo0aMN8fHxrfaTyrXCOfAiycrKgkKhMJuaIQgC4uLiUFhYiIKCgha33blzJ4YPH242BcDd3R1jxoxBVlaWRcfdnXUkJmQ5Mln7/jdVU1ODH374AY8//rjZ06u+ffti0KBByMzM7Kwh9kjtjQtZjqenZ5M2pVKJoKAgXLp0qdVteV+xjI7EhLqWXC6HQqG47284pHKt8P/AIsnPz4dGo2lyEzTOicvLy2t2u1u3bqG4uBghISFNvgsNDUV5eTnKy8s7f8A9QHtj0tiECRPQr18/jBw5Eu+88w5jIaLz58/j1q1beOihh5p8Fxoaivz8fBFGRUY3b97E8OHD0a9fP4wZMwbLli1DVVWV2MPqdioqKpCfn9/sdWDE+0rXaktMGuN9xXL0ej1qa2tx+fJlrFq1Cr/99huef/75FvtL6VrhHHiRaLVa9OnTp0m7q6ur6fvmVFZWwmAwmPo15ubmZtq2uZ/6qXXtjQkABAYG4rXXXkO/fv1ga2uLI0eOYP369di/fz+2bt3abLzIsozxaulauXXrFm7dugUHB4euHlqPFxYWhrCwMISEhKCurg45OTnYuHEjDh06hM2bN3OObycxGAxISkqCXq/H7NmzW+zH+0rXaWtMAN5XusKrr76K77//HgDg4uKCf/zjHxg9enSL/aV0rTCBF5EgCO36ri3fU/u0NyaTJ082+xwdHY3IyEjMmjULmzZtwrx58zptjPRgOnKdkWW88MILZp9HjRoFtVqNpKQkfPfdd5g0aZI4A+tm/va3v2Hnzp1YunQp+vbte9/+vB4s70FiwvuK5S1cuBBz5sxBWVkZMjIy8Oqrr2LZsmWYOHFiq9tJ4VrhFBqRuLm5NftEt7KyEkDzTw2N7YIgNLutsc34UyA9mPbGpCUjRoyASqXCsWPHOmV89GAaPw25l1arhYODA+zt7bt6WNSCp556CjKZjNdLJ0lOTsbHH3+MRYsWYcqUKa325X2lazxITFrC+0rnCgwMxMCBAzF27FisXLkSI0eOxHvvvQe9Xt9sfyldK0zgRaLRaHD27Nkmf0mM86ybm18FAA4ODggMDGx2PnZeXh48PDz4a852am9MWmMwGLjYTySBgYFwcHBodq57Xl5em+efUtcwGAwAuDi2M3zwwQdYs2YNFi5ciOeee+6+/XlfsbwHjUlreF+xnIiICFRWVqKioqLZ76V0rfBvgEhiY2Oh0+mavDs5PT0darW61WIA48ePR05ODq5evWpq02q12LNnD2JjYy025u6uIzFpzt69e1FWVoZBgwZ15jCpjWxtbRETE4Pvv/8e1dXVpvaioiIcO3YMjz32mIijo3tt27YNer2e10sHpaSkIDU1FYmJiZgzZ06bt+N9xXLaG5Pm8L5iOQaDAbm5uVAqla0+RZfKtcI58CKJiYlBVFQUFi1aBK1Wi4CAAKSnp+PPtC+GAAAG9klEQVTw4cNITU019YuPj0dubi7OnDljaps9eza2bduGuXPnYv78+ZDL5fjoo48gl8vx0ksviXE63UJHYjJ58mRMnjwZarUacrkcR48exb/+9S8EBQVhxowZYpxOt7Jjxw4AwIkTJwAAP//8M65duwZHR0dTga2xY8cCgNkPYK+88gqmTp2Kl19+GbNmzTIVcvL398f06dO7+Cy6n/bE5eLFi3jjjTfwxBNPoHfv3qirq8P+/fvx2WefYfDgwZgwYYIIZ9I9fPzxx1i9ejXGjBmD4cOHm02zsLOzQ//+/QHwvtKVOhIT3lcs509/+hP8/f0RHh4Od3d3XL16FWlpaThw4ACSkpIgl9enx1K+VpjAi0QQBKSmpmLlypVITk6GTqeDRqNBSkqK6YbXEi8vL2zatAnLly/HG2+8AYPBgCFDhuCzzz6Dn59fF51B99ORmAQHB+Pzzz/HlStXUFtbCx8fH0ydOhXz5s2DUqnsojPovhITE80+r169GgDg7+/fagVQjUaDTz/9FO+//z5eeeUVyOVyjBgxAm+99RZcXFwsOuaeoD1xcXFxgbu7O9avX4+ysjIYDAYEBgZi7ty5mDt3runGSQ9uz549pn8b/9voftcK7yuW0ZGY8L5iOYMHD8Y333yDL7/8EtevX4dCocCAAQPw0UcfWU0OJhiMEw+JiIiIiEjyOAeeiIiIiMiKMIEnIiIiIrIiTOCJiIiIiKwIE3giIiIiIivCBJ6IiIiIyIowgSciIiIisiJM4ImISPLi4+Pv+35mIqKeghUziIh6qIMHD+K5555r8XsbGxucOnWqC0dERERtwQSeiKiHmzhxIkaPHt2kXSbjL2mJiKSICTwRUQ/Xv39/TJo0SexhEBFRG/HxChERterChQsIDQ3F6tWrkZGRgSeffBIRERF49NFHsXr1atTW1jbZ5vTp05g/fz6ioqIQERGBCRMm4J///Cfq6uqa9L169SoWL16McePGYcCAAYiOjsYf/vAH7Nu3r0nfy5cv47XXXsOwYcMQGRmJ2bNno6ioyCLnTUQkVXwCT0TUw1VXV6OioqJJu52dHVxcXEyf9+zZg08//RQzZsyAl5cXdu/ejZSUFJSUlGDp0qWmfidOnEB8fDzkcrmp7549e/D+++/j9OnT+Pvf/27qe+HCBfz+979HeXk5Jk2ahAEDBqC6uhrHjx9HTk4ORowYYep78+ZNzJw5E4MGDcIf//hHXLhwARs2bMC8efOQkZEBGxsbC/0JERFJCxN4IqIebvXq1Vi9enWT9kcffRRr1641ff7111+xZcsWhIeHAwBmzpyJhIQEbN26FdOmTUNkZCQAYMmSJbhz5w6++OILhIWFmfq++uqryMjIwDPPPIPo6GgAwP/93//hypUrWL9+PUaNGmV2fL1eb/b52rVrmD17Nl588UVTm4eHB1asWIGcnJwm2xMRdVdM4ImIerhp06bh8ccfb9Lu4eFh9nn48OGm5B0ABEHAnDlzsHPnTmRlZSEyMhLl5eU4evQoYmNjTcm7se9LL72EHTt2ICsrC9HR0dBqtfjpp58watSoZpPvexfRymSyJm/NeeSRRwAA586dYwJPRD0GE3gioh4uKCgIw4cPv2+/vn37NmnTaDQAgPPnzwOonxLTuP3e7WUymalvcXExDAYD+vfv36Zx9urVC/b29mZtbm5uAACtVtumfRARdQdcxEpERG0iCMJ9+xgMhjbvz9i3LfsF0Ooc9wc5LhGRtWMCT0REbVJQUNBiW2BgoNm/m+tbWFgIvV5v6hMUFARBEFgsiojoATGBJyKiNsnJycEvv/xi+mwwGLB+/XoAwPjx4wEAnp6eGDx4MPbs2YO8vDyzvuvWrQMAxMbGAqif/jJ69Gj8+OOPyMnJaXI8PlUnImoe58ATEfVwp06dwtdff93sd8bEHADCwsLw/PPPY8aMGVCpVNi1axdycnIwadIkDB482NRv0aJFiI+Px4wZMzB9+nSoVCrs2bMHe/fuxcSJE01voAGApKQknDp1Ci+++CImT56M8PBw3L59G8ePH4e/vz8WLlxouRMnIrJSTOCJiHq4jIwMZGRkNPtdZmamae752LFjoVarsXbtWhQVFcHT0xPz5s3DvHnzzLaJiIjAF198gVWrVmHz5s24efMmAgMD8frrr2PWrFlmfQMDA/HVV1/hww8/xI8//oivv/4aSqUSYWFhmDZtmmVOmIjIygkG/o6SiIhaceHCBYwbNw4JCQlYsGCB2MMhIurxOAeeiIiIiMiKMIEnIiIiIrIiTOCJiIiIiKwI58ATEREREVkRPoEnIiIiIrIiTOCJiIiIiKwIE3giIiIiIivCBJ6IiIiIyIowgSciIiIisiJM4ImIiIiIrMj/A7eMS/sKtUQoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 训练过程的损失变化\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "import seaborn as sns\n",
    "\n",
    "# Use plot styling from seaborn.\n",
    "sns.set(style='darkgrid')\n",
    "\n",
    "# Increase the plot size and font size.\n",
    "sns.set(font_scale=1.5)\n",
    "plt.rcParams[\"figure.figsize\"] = (12,6)\n",
    "\n",
    "# Plot the learning curve.\n",
    "plt.plot(loss_values, 'b-o')\n",
    "\n",
    "# Label the plot.\n",
    "plt.title(\"Training loss\")\n",
    "plt.xlabel(\"Epoch\")\n",
    "plt.ylabel(\"Loss\")\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 评估模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:41:46.654431Z",
     "start_time": "2020-05-05T14:41:46.570078Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of test sentences: 516\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 测试集数据处理\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "# Load the dataset into a pandas dataframe.\n",
    "df = pd.read_csv(\"../datasets/cola_public//raw/out_of_domain_dev.tsv\",\n",
    "                 delimiter='\\t',\n",
    "                 header=None,\n",
    "                 names=['sentence_source', 'label', 'label_notes', 'sentence'])\n",
    "\n",
    "# Report the number of sentences.\n",
    "print('Number of test sentences: {:,}\\n'.format(df.shape[0]))\n",
    "\n",
    "# Create sentence and label lists\n",
    "sentences = df.sentence.values\n",
    "labels = df.label.values\n",
    "\n",
    "# 文本预处理\n",
    "# Tokenize all of the sentences and map the tokens to thier word IDs.\n",
    "input_ids = []\n",
    "\n",
    "# For every sentence...\n",
    "for sent in sentences:\n",
    "    # `encode` will:\n",
    "    #   (1) Tokenize the sentence.\n",
    "    #   (2) Prepend the `[CLS]` token to the start.\n",
    "    #   (3) Append the `[SEP]` token to the end.\n",
    "    #   (4) Map tokens to their IDs.\n",
    "    encoded_sent = tokenizer.encode(\n",
    "        sent,  # Sentence to encode.\n",
    "        add_special_tokens=True,  # Add '[CLS]' and '[SEP]'\n",
    "    )\n",
    "\n",
    "    input_ids.append(encoded_sent)\n",
    "\n",
    "# Pad our input tokens\n",
    "input_ids = pad_sequences(input_ids,\n",
    "                          maxlen=MAX_LEN,\n",
    "                          dtype=\"long\",\n",
    "                          truncating=\"post\",\n",
    "                          padding=\"post\")\n",
    "\n",
    "# Create attention masks\n",
    "attention_masks = []\n",
    "\n",
    "# Create a mask of 1s for each token followed by 0s for padding\n",
    "for seq in input_ids:\n",
    "    seq_mask = [float(i > 0) for i in seq]\n",
    "    attention_masks.append(seq_mask)\n",
    "\n",
    "# Convert to tensors.\n",
    "prediction_inputs = torch.tensor(input_ids)\n",
    "prediction_masks = torch.tensor(attention_masks)\n",
    "prediction_labels = torch.tensor(labels)\n",
    "\n",
    "# Set the batch size.\n",
    "batch_size = 32\n",
    "\n",
    "# Create the DataLoader.\n",
    "prediction_data = TensorDataset(prediction_inputs, prediction_masks,\n",
    "                                prediction_labels)\n",
    "prediction_sampler = SequentialSampler(prediction_data)\n",
    "prediction_dataloader = DataLoader(prediction_data,\n",
    "                                   sampler=prediction_sampler,\n",
    "                                   batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:41:49.953158Z",
     "start_time": "2020-05-05T14:41:49.386526Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicting labels for 516 test sentences...\n",
      "    DONE.\n"
     ]
    }
   ],
   "source": [
    "# 在测试集上进行预测\n",
    "\n",
    "print('Predicting labels for {:,} test sentences...'.format(\n",
    "    len(prediction_inputs)))\n",
    "\n",
    "# Put model in evaluation mode\n",
    "model.eval()\n",
    "\n",
    "# Tracking variables\n",
    "predictions, true_labels = [], []\n",
    "\n",
    "# Predict\n",
    "for batch in prediction_dataloader:\n",
    "    # Add batch to GPU\n",
    "    batch = tuple(t.to(device) for t in batch)\n",
    "\n",
    "    # Unpack the inputs from our dataloader\n",
    "    b_input_ids, b_input_mask, b_labels = batch\n",
    "\n",
    "    # Telling the model not to compute or store gradients, saving memory and\n",
    "    # speeding up prediction\n",
    "    with torch.no_grad():\n",
    "        # Forward pass, calculate logit predictions\n",
    "        outputs = model(b_input_ids,\n",
    "                        token_type_ids=None,\n",
    "                        attention_mask=b_input_mask)\n",
    "\n",
    "    logits = outputs[0]\n",
    "\n",
    "    # Move logits and labels to CPU\n",
    "    logits = logits.detach().cpu().numpy()\n",
    "    label_ids = b_labels.to('cpu').numpy()\n",
    "\n",
    "    # Store predictions and true labels\n",
    "    predictions.append(logits)\n",
    "    true_labels.append(label_ids)\n",
    "\n",
    "print('    DONE.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:42:20.366783Z",
     "start_time": "2020-05-05T14:42:20.361083Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Positive samples: 354 of 516 (68.60%)\n"
     ]
    }
   ],
   "source": [
    "print('Positive samples: %d of %d (%.2f%%)' %\n",
    "      (df.label.sum(), len(df.label),\n",
    "       (df.label.sum() / len(df.label) * 100.0)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-05T14:44:01.949475Z",
     "start_time": "2020-05-05T14:44:01.942164Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MCC: 0.524\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import matthews_corrcoef\n",
    "\n",
    "matthews_set = []\n",
    "\n",
    "# Evaluate each test batch using Matthew's correlation coefficient\n",
    "print('Calculating Matthews Corr. Coef. for each batch...')\n",
    "\n",
    "# For each input batch...\n",
    "for i in range(len(true_labels)):\n",
    "\n",
    "    # The predictions for this batch are a 2-column ndarray (one column for \"0\"\n",
    "    # and one column for \"1\"). Pick the label with the highest value and turn this\n",
    "    # in to a list of 0s and 1s.\n",
    "    pred_labels_i = np.argmax(predictions[i], axis=1).flatten()\n",
    "\n",
    "    # Calculate and store the coef for this batch.\n",
    "    matthews = matthews_corrcoef(true_labels[i], pred_labels_i)\n",
    "    matthews_set.append(matthews)\n",
    "\n",
    "# Combine the predictions for each batch into a single list of 0s and 1s.\n",
    "flat_predictions = [item for sublist in predictions for item in sublist]\n",
    "flat_predictions = np.argmax(flat_predictions, axis=1).flatten()\n",
    "\n",
    "# Combine the correct labels for each batch into a single list.\n",
    "flat_true_labels = [item for sublist in true_labels for item in sublist]\n",
    "\n",
    "# Calculate the MCC\n",
    "mcc = matthews_corrcoef(flat_true_labels, flat_predictions)\n",
    "\n",
    "print('MCC: %.3f' % mcc)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">Matthews相关系数，用于度量二分类的质量。它会考虑TP/FP/TN/FP的情况，通常被认为是一个balanced的度量 ，可以用于那些有着不同size的分类中。MCC本质上是一个介于［－1，+1］之间的相关系数值。相关系数为+1，表示是一个完美的预测，0表示是一个平均随机预测（average random prediction），而-1表示是一个逆预测（inverse prediction）。这种统计方法也被称为：phi coefficient。   \n",
    "\n",
    "$$MCC = \\frac{tp \\times tn - fp \\times fn}{\\sqrt{(tp + fp)(tp + fn)(tn + fp)(tn + fn)}}.$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BERT用于问答\n",
    "给定`question`文本和包含该问题答案的对应的`answer_text`文本，从中找出`question`的答案"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-06T07:10:04.882237Z",
     "start_time": "2020-05-06T07:09:59.583434Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Calling BertTokenizer.from_pretrained() with the path to a single file or url is deprecated\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "from transformers import BertForQuestionAnswering, BertTokenizer\n",
    "\n",
    "model_path = \"../../H/models/huggingface/bert-large-uncased-whole-word-masking-\\\n",
    "finetuned-squad/\"\n",
    "tokenizer = BertTokenizer.from_pretrained(model_path + \"vocab.txt\")\n",
    "model = BertForQuestionAnswering.from_pretrained(model_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-06T07:12:17.629916Z",
     "start_time": "2020-05-06T07:12:17.624104Z"
    }
   },
   "outputs": [],
   "source": [
    "question = \"How many parameters does BERT-large have?\"\n",
    "answer_text = \"BERT-large is really big...it has 24-layers and an embedding size \\\n",
    "of 1024, for a total of 340M parameters!\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-06T07:12:18.139563Z",
     "start_time": "2020-05-06T07:12:18.133633Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The input has a total of 44 tokens.\n"
     ]
    }
   ],
   "source": [
    "input_ids = tokenizer.encode(question, answer_text)\n",
    "print(\"The input has a total of {:} tokens.\".format(len(input_ids)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-06T07:14:13.242438Z",
     "start_time": "2020-05-06T07:14:13.234660Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CLS]           101\n",
      "how           2,129\n",
      "many          2,116\n",
      "parameters   11,709\n",
      "does          2,515\n",
      "bert         14,324\n",
      "-             1,011\n",
      "large         2,312\n",
      "have          2,031\n",
      "?             1,029\n",
      "[SEP]           102\n",
      "bert         14,324\n",
      "-             1,011\n",
      "large         2,312\n",
      "is            2,003\n",
      "really        2,428\n",
      "big           2,502\n",
      ".             1,012\n",
      ".             1,012\n",
      ".             1,012\n",
      "it            2,009\n",
      "has           2,038\n",
      "24            2,484\n",
      "-             1,011\n",
      "layers        9,014\n",
      "and           1,998\n",
      "an            2,019\n",
      "em            7,861\n",
      "##bed         8,270\n",
      "##ding        4,667\n",
      "size          2,946\n",
      "of            1,997\n",
      "102           9,402\n",
      "##4           2,549\n",
      ",             1,010\n",
      "for           2,005\n",
      "a             1,037\n",
      "total         2,561\n",
      "of            1,997\n",
      "340          16,029\n",
      "##m           2,213\n",
      "parameters   11,709\n",
      "!               999\n",
      "[SEP]           102\n"
     ]
    }
   ],
   "source": [
    "tokens = tokenizer.convert_ids_to_tokens(input_ids)\n",
    "for token, id in zip(tokens, input_ids):\n",
    "    print(\"{:<12} {:>6,}\".format(token, id))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-06T07:16:48.788163Z",
     "start_time": "2020-05-06T07:16:48.782382Z"
    }
   },
   "outputs": [],
   "source": [
    "# segment mask\n",
    "\n",
    "sep_index = input_ids.index(tokenizer.sep_token_id)\n",
    "\n",
    "num_seg_a = sep_index + 1\n",
    "num_seg_b = len(input_ids) - num_seg_a\n",
    "\n",
    "segment_ids = [0] * num_seg_a + [1] * num_seg_b\n",
    "\n",
    "assert len(segment_ids) == len(input_ids)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-06T07:19:02.867390Z",
     "start_time": "2020-05-06T07:19:02.727443Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([1, 44]), torch.Size([1, 44]))"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 输入为一维，即无批数据维度时，不需要 pad \n",
    "\n",
    "start_scores, end_scores = model(torch.tensor([input_ids]),\n",
    "                                 token_type_ids=torch.tensor([segment_ids]))\n",
    "start_scores.shape, end_scores.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-06T07:21:12.986952Z",
     "start_time": "2020-05-06T07:21:12.980320Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer: 340 ##m\n"
     ]
    }
   ],
   "source": [
    "# 选择策略，不仅仅：是单个向量中最大的\n",
    "answer_start = torch.argmax(start_scores)\n",
    "answer_end = torch.argmax(end_scores)\n",
    "answer = \" \".join(tokens[answer_start:answer_end + 1])\n",
    "print(\"Answer: \" + answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-06T07:28:00.033201Z",
     "start_time": "2020-05-06T07:28:00.027319Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer: 340m\n"
     ]
    }
   ],
   "source": [
    "# 将 子词 重新组合成 单词\n",
    "answer = tokens[answer_start]\n",
    "for i in range(answer_start + 1, answer_end + 1):\n",
    "    if tokens[i][0:2] == '##':\n",
    "        answer += tokens[i][2:]\n",
    "    else:\n",
    "        anser += \" \" + tokens[i]\n",
    "print(\"Answer: \" + answer)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-06T07:39:37.019259Z",
     "start_time": "2020-05-06T07:39:36.662762Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show(*args, **kw)>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6sAAAI0CAYAAAD7pb+nAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdeZQU9b0/7vcsoChEFgcUcSVRNIqgIGhcIwmLqCCiuF3jEkUjZjGuiaKSqGgS48813gTMVaPGDRXFNUa96jXAUQlRE3e9Yd8HZJitfn/4ZS4wgHT31FDDPM85nDN0db0+n67p6u7XdFV3UZIkSQAAAECGFG/sCQAAAMCalFUAAAAyR1kFAAAgc5RVAAAAMkdZBQAAIHOUVQAAADJHWQWAJuR///d/Y7fddovq6uqNPRUASJWyCkCTNWXKlBgxYkTsu+++sd9++8WIESNi2rRpERHxyCOPxAknnFBQ/lcVwzlz5sRuu+0W8+bNq7vs9ttvX+tlZ5xxRkFz2VDr2yYA0JQoqwA0SUuXLo2RI0fGySefHH/729/i5ZdfjvPOOy9atmzZIPkb8s5lx44dY8cdd4zJkyfXXTZlypTYZZdd6l3Wu3fvVOawqrS3yUo1NTUNmgcAa6OsAtAkffzxxxERMXjw4CgpKYnNN988DjzwwOjWrVt8+OGHMXr06HjrrbeiZ8+e0atXr4iI+Otf/xpDhgyJffbZJw455JC4+eab6/JWvov64IMPxqGHHhqnnnpqnHzyyRER0bt37+jZs2e8+eab9ebRq1evmDJlSkR8WeLeeeed+I//+I/VLnvzzTfr5lBeXh4XXXRR9O3bNw477LC47bbbora2NiK+fDd4xIgRcc0118R+++0XN998c9TU1MTYsWOjT58+cfjhh8dLL72U1zZZ6c9//nMMHDgwevbsGYMGDYp//OMfERHx4YcfximnnBK9evWKI444Il544YW6dS655JIYPXp0fP/7348ePXrEG2+8EZWVlTF27Ng49NBD44ADDogrrrgiKioqIiJiwYIFcfbZZ0evXr1iv/32ixNPPLHuNgLAhird2BMAgHzsvPPOUVJSEhdffHEMGjQoevToEVtttVVERHTt2jWuuuqqePDBB+O+++6rW6dVq1YxduzY+MY3vhH/+te/4vTTT4/dd989+vXrV3edyZMnx1NPPRXFxcUxb968OPzww2Py5MlRWrr2p8zevXvH+PHjIyLinXfeiV122SX233//unHfeeedqK6uju7du0dExJgxY6K8vDyef/75WLRoUZxxxhlRVlYWw4cPj4iIadOmxRFHHBGvvfZaVFdXx5///Od48cUXY8KECdGqVasYNWpUXtskImLSpElx8803x6233hp77bVXfPbZZ1FaWhpVVVUxcuTIGDZsWPzhD3+IqVOnxrnnnhsPP/xw7LLLLhERMXHixLjzzjvjd7/7XVRVVcUNN9wQn3/+eUyYMCFKS0vjpz/9adx6661xwQUXxPjx46NTp07x+uuvR0TE22+/HUVFRbn9ggFo9ryzCkCT1Lp16/jTn/4URUVFcfnll8f+++8fI0eOXO1c0TX16dMndttttyguLo5u3brFEUccEX/7299Wu86oUaNiiy22iM0333yD5tG7d+94//33Y/HixTF16tTo1atX7LTTTrFw4cK6y/bee+9o2bJl1NTUxFNPPRUXXHBBtG7dOrp06RKnnXZaPP7443V5HTt2jFNOOSVKS0tj8803j0mTJsWpp54a2267bbRt2zbOPvvsvLfJQw89FGeeeWZ07949ioqKYscdd4ztttsu3n777fjiiy/irLPOipYtW8b+++8fhx12WDz55JN12Ycffnjsu+++UVxcHC1btowHH3wwLrvssmjbtm20bt06zj777Lrrl5aWxty5c2PGjBnRokWL6NWrl7IKQM6UVQCarK5du8Z1110XL7/8cjzxxBMxZ86cuOaaa9Z5/bfffjtOOeWU6Nu3b+y7775x//33x8KFC1e7zjbbbJPTHLp06RLbbLNNTJ06NSZPnhz77rtvRET07Nmz7rKV56suXLgwqqqqonPnznXrd+7cOWbPnr3O8efMmRPbbrvtatdfn/Vtk5kzZ8YOO+xQb505c+bENttsE8XF//eyYM15rTqHBQsWxPLly+OYY46JXr16Ra9eveLMM8+s25ZnnHFG7LjjjnH66afH4YcfHnfeeed65wwAa6OsArBJ6Nq1axxzzDHx/vvvR0Ss9Z28Cy64oO68z6lTp8aIESMiSZLVrrPqehv6buC+++4bkydPjrfeeiv22Wef1S6bOnVqXVlt165dtGjRImbMmFG37syZM6NTp07rHLOsrCxmzpy52vU31JrbZNttt43PPvus3vU6duwYs2bNWu280jXntap27drF5ptvHk8++WRMmTIlpkyZElOnTq07p7d169ZxySWXxAsvvBB33HFHjB8/vu6QYADYUMoqAE3Shx9+GOPGjYtZs2ZFxJflauLEibH33ntHRESHDh1i9uzZUVlZWbfOsmXLYquttorNNtsspk2bFhMnTlzvGO3bt4/i4uL4/PPP13u93r17x2OPPRYdO3aM1q1bR8SXZfWxxx6LpUuXRo8ePSIioqSkJAYMGBA33nhjLF26NP7973/H+PHj46ijjlpn9sCBA+Puu++OWbNmxeLFi9f7LuVXbZNjjz02xo0bF9OnT48kSeLTTz+Nf//739G9e/do1apV/P73v4+qqqp444034i9/+UsMGjRoreMUFxfH8OHD45prron58+dHRMTs2bPjlVdeiYiIF198MT799NNIkiRat24dJSUlq71rCwAbwgcsAdAktW7dOt5+++0YP358lJeXR5s2beKwww6Liy66KCIi+vbtG1//+tfjwAMPjKKionjjjTdi9OjRMXbs2Lj66qtjv/32i4EDB8aSJUvWOUarVq1i5MiRccIJJ0R1dXX8/ve/ryueq+rdu3fMnz9/tXK3++67R0VFRXzzm9+MVq1a1V1++eWXx5gxY6Jfv36x2WabxfDhw2PYsGHrnMNxxx0Xn3zySRx99NGx5ZZbxhlnnBH/8z//k9c2GThwYCxatCguuOCCmDNnTmy33XZx/fXXx3bbbRe33357XHXVVfG73/0uOnXqFNdff3107dp1nfO68MIL49Zbb43jjjsuFi5cGJ06dYoTTjghDjrooPj0009jzJgxsWDBgvja174WJ5xwQvTp02edWQCwNkXJmsc/AQAAwEbmmBwAAAAyR1kFAAAgc5RVAAAAMkdZBQAAIHOUVQAAADJHWQUAACBzmsT3rC5cuCxqa9f/DTsdOrSO+fOXNtiYDZ2XRmbW89LIbG55aWRmPS+NzOaWl0Zm1vPSyGxueWlkZj0vjcys56WR2dzy0sjMel4amc0tL43MrOdtaGZxcVG0a7flWpc1ibJaW5t8ZVldeb2GHrehZX2ObnP28tLIzHpeGpnNLS+NzKznpZHZ3PLSyMx6XhqZWc9LI7O55aWRmfW8NDKbW14amVnPKzTTYcAAAABkjrIKAABA5iirAAAAZI6yCgAAQOYoqwAAAGSOsgoAAEDmKKsAAABkjrIKAABA5iirAAAAZI6yCgAAQOYoqwAAAGSOsgoAAEDmKKsAAABkjrIKAABA5iirAAAAZI6yCgAAQOYoqwAAAGRO6caeAAAAQGNqv9UWUdKyZL3XKStrs97lNZU1sWDxFw05LdagrAIAAM1KScuSmPWbvxeUsc1P9mqg2bAuDgMGAAAgc5RVAAAAMkdZBQAAIHOUVQAAADKnUT5gacWKFXHNNdfE66+/Hptttln06NEjxowZ0xhDAwAA0AQ1Slm94YYbYrPNNotnnnkmioqKYt68eY0xLAAAAE1U6mV12bJlMWHChHjppZeiqKgoIiK23nrrtIcFAACgCUv9nNXPP/882rZtG7fcckscc8wxccopp8SUKVPSHhYAAIAmrChJkiTNAaZPnx7Dhg2LX/3qV3HkkUfG22+/HSNHjoznnnsuWrdunebQAAAAazXrN38vaP1tfrJXA82EdUn9MODOnTtHaWlpDB48OCIi9t5772jXrl18/PHHsddeG/YLnj9/adTWrr9Tl5W1iblzywueb1p5aWRmPS+NzOaWl0Zm1vPSyGxueWlkZj0vjczmlpdGZtbz0sjMel4amc0tL43MrOelkbmx88rK2jTIuLmOuSltw4bKLC4uig4d1v4mZuqHAbdv3z769OkTr776akREfPzxxzF//vzYcccd0x4aAACAJqpRPg34qquuissuuyzGjh0bpaWlcf3118fXvva1xhgaAACAJqhRyur2228fd999d2MMBQAAwCYg9cOAAQAAIFfKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOY1aVm+55ZbYbbfd4l//+ldjDgsAAEAT02hl9R//+Ee89dZb0blz58YaEgAAgCaqUcpqZWVlXH311TF69OgoKipqjCEBAABowhqlrN50001x1FFHxfbbb98YwwEAANDEFSVJkqQ5wJtvvhk33nhj/PGPf4yioqL49re/HXfccUfsuuuuaQ4LAACwTrN+8/eC1t/mJ3s10ExYl9K0B5g8eXJ89NFHcfjhh0dExKxZs+KMM86Ia6+9Ng488MANypg/f2nU1q6/U5eVtYm5c8sLnm9aeWlkZj0vjczmlpdGZtbz0shsbnlpZGY9L43M5paXRmbW89LIzHpeGpnNLS+NzKznpZG5sfPKyto0yLi5jrkpbcOGyiwuLooOHVqvdVnqZfWss86Ks846q+7/3lkFAADgq/ieVQAAADIn9XdW1/SXv/ylsYcEAACgifHOKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDmlaQ+wcOHCuOiii+Kzzz6Lli1bxo477hhXX311tG/fPu2hAQAAaKJSf2e1qKgozjzzzHjmmWfiiSeeiO233z5+9atfpT0sAAAATVjqZbVt27bRp0+fuv/36NEjZsyYkfawAAAANGGNes5qbW1t3HffffHtb3+7MYcFAACgiSlKkiRprMGuuuqqmD17dtxyyy1RXOyznQAAgI1j1m/+XtD62/xkrwaaCeuS+gcsrTR27Nj49NNP44477si5qM6fvzRqa9ffqcvK2sTcueWFTDHVvDQys56XRmZzy0sjM+t5aWQ2t7w0MrOel0Zmc8tLIzPreWlkZj0vjczmlpdGZtbz0sjc2HllZW0aZNxcx9yUtmFDZRYXF0WHDq3XuqxRyuqNN94Y06dPjzvvvDNatmzZGEMCAADQhKVeVt9///244447YqeddooRI0ZERESXLl3i1ltvTXtoAAAAmqjUy+o3vvGN+Oc//5n2MAAAAGxCfMoRAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZ0yhl9eOPP47jjz8++vfvH8cff3x88sknjTEsAAAATVSjlNXRo0fHiSeeGM8880yceOKJccUVVzTGsAAAADRRqZfV+fPnxzvvvBODBw+OiIjBgwfHO++8EwsWLEh7aAAAAJqo1MvqzJkzo1OnTlFSUhIRESUlJdGxY8eYOXNm2kMDAADQRBUlSZKkOcD06dPj4osvjieffLLuskGDBsUNN9wQ3/zmNzc4J6muiaLSkrznseb6SXV1FJWW5p23toxCM+vnVUVRaYsC8lZfv7a6MopLW+adt+b6healkbnm+jXVlVFSQN6a6xeal0Zm2nnVNZVRWpJ/3prrF5qXRuaa61fVVEaLAvLWXL+ypjJaFnib18yorKmKliX5Pz6suX7D51VHy5L8Hw/Xtn5DZ2Y/ryZaluT/vLe2jEIzm1peOnOsjZYl+f+tf831Gz4viZYlRXnnrS2juiaJ0gIy11y/ofNqapIoKfA2r5lRaOaa69fWJFFcQN6a69dWJ1FcWkDeGusXmre2jKQ6iaICMtdcP6mujaLS/PeVNdcvNC+NzPTzGrpPFZa3IRmFtbUNsO2228bs2bOjpqYmSkpKoqamJubMmRPbbrvtBmfMn780OnRoHXNvvyfveZSdc3LMnVv+f/8vaxNz7/hd3nkREWUjz66XOfv26/LO63TOJfXyZtz6o7zzOv/gt/XyPrz56Lzzuo56rF7e27cflXdeRMTe5zxeL/O1OwfnnXfAWRPr5b3w+yPyzjv8zCfr5U0cNzDvvIiIwadPqpf54PgBeecNP+3penl/vOu7eeed+r1n6+X97u7+eeedfcoz9fJu/FP+eRERPz6xfuaYB/LPvPz4+nk/eTj/38lvhtX/nRz7WP55EREPHV0/c+BjI/POm3T0HfXyBk34ed55Tw35Rf28R/N/PHxq6OqPhyszj3jkprwznzzmh/XmeMTD+T8PPDms/nPA4IfH5503cdhp9fMeujfvvIiIiceetJbMBwvIG14v76iHHss77/Fjj66Xd/RDT+ed99ixA9Z6vxn68F/zznx02KH15jjs4Tfyznt4WJ96ecMfnp533oPD9qyXd+ojn+adFxHxx2N2rP8Y++iMvPMuH9q5/vPKI3Pyzjv7mI718v788Ly88yIijhu2db3MSQ/knznw+Pp5L989N++8g08pq5c39Q/5b8N9z6i/Dd+7bXbeeRER3c7tVC/zf389K++8LhdsUy9v1q8+yjtvm5/uUu/xYX3KytrkdP2V68z+7d9ynVqdTj/ar363+P9eyj/v/EPq5c25Jf/H2I7nDaifd+ujeedFRHT8wdC6rrc2qR8G3KFDh9h9991j4sSJERExceLE2H333aN9+/ZpDw0AAEATlfo7qxERV155ZVxyySVx2223xde+9rUYO3ZsYwwLAMAmrqqqNo4btnXBGUD2NEpZ7dq1azz4YP6HHQEAwNosWrRsvcvzOZwTyIZG+Z5VAAAAyIWyCgAAQOYoqwAAAGROo5yzCgAAZFN1ZW10O7dTwRnQ0JRVAABoxhYuXv+HVEX4oCo2DocBAwAAkDnKKgAAAJmjrAIAAJA5zlkFAAAaVE1lbXS5YJuC1gdlFQAAaFALvuJDm3xgExvCYcAAAABkjrIKAABA5iirAAAAZI6yCgAAQOYoqwAAAGSOsgoAAEDm+OoaAMigiqqqmHjsSQVnAEBTpawCQAaVL6qI8qhY53LfUQjAps5hwAAAAGSOsgoAAEDmKKsAAABkjrIKAABA5iirAAAAZI6yCgAAQOYoqwAAAGSOsgoAAEDmKKsAAABkjrIKAABA5iirAAAAZI6yCgAAQOYoqwAAAGSOsgoAAEDmKKsAAABkjrIKAABA5iirAAAAZI6yCgAAQOYoqwAAAGSOsgoAAEDmKKsAAABkjrIKAABA5iirAAAAZI6yCgAAQOaUphl+1VVXxeuvvx4tW7aMLbbYIn72s5/FXnvtleaQAAAAbAJSfWf14IMPjieeeCIef/zxOPvss+PHP/5xmsMBAACwiUj1ndXDDjus7ucePXrErFmzora2NoqLHX0MAADAujVaa7z33nvj0EMPVVQBAAD4SgW9szp06NCYMWPGWpe99tprUVJSEhERTz75ZDzxxBNx77335jVOhw6t857jqsrK2jRITpqZzS0vjcys56WR2dzy0sjMel4amc0tL43MjZ2XxjZq7DGznpdGZtbz0sjc2Hn2lYbJy8IcGnM895v08yLW3/UKKquPPvroV17nueeeixtvvDHuuuuu2HrrrfMaZ/78pQ1SWOfOLa/7uaE2dENnNre8NDKznpdGZnPLSyMz63lpZDalvDQys573VcrK2uR0/ZXrFKqpbcOszzGLeWlk2leynbch8tmODZnnfrNp5kWsv+ules7qiy++GNdee22MHz8+unTpkuZQAAAAbEJSLauXXnpptGjRIs4///y6y+66665o165dmsMCAADQxKVaVv/nf/4nzXgAAAA2UT6aFwAAgMxRVgEAAMgcZRUAAIDMUVYBAADInFQ/YAkAAGhY1ZW1se8ZHQtaH5oCZRUAAJqQhYuXrXd5WVmbmDu3vJFmA+lxGDAAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5vroGgGapoqoqnhx2dkHrAwDpUVYBaJbKF1VEeVSsc7nvKQSAjUtZBQAAMq2msia2+ekuBa2ftprKmuj0o/0KWp/VKasAAECmLVj8xXqXZ+FomKYwx6bGBywBAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZU7qxJwBkW2VVRZx9yjMFrQ8AALlSVoH1WryoKiKq1rm8rKxNzJ1b3ngTAgCgWXAYMAAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmdMoZfWNN96I3XffPe65557GGA4AAIAmLvWyunTp0vjVr34VBx98cNpDAQAAsIlIvaxed911ccYZZ0S7du3SHgoAAIBNRKpl9aWXXoolS5bEgAED0hwGAACATUxpISsPHTo0ZsyYsdZlTz/9dPz617+O8ePHFzJERER06NC64IyIiLKyNg2Sk2Zmc8tLIzPreWlkbuy8NLZRY4+Z9bw0MptbXj6ZjX3fzvr80hgz63lpZGY9L43MjZ3XHPcV94vs5aWV2ZjjpTH/9XW9gsrqo48+us5lU6ZMiblz58bw4cMjImLhwoXx4osvxqJFi+K8887LaZz585c2SGGdO7e87ueG2tANndnc8tLIzHpeGplp5n2VsrI2OV+/ITSlbeg2F56XVub6bIz7dprzW7lOoZr7/aY55KWR2ZSep1auU6imdpsbO7O55eWTmfX7YUM93qyv6xVUVtenV69e8frrr9f9/5JLLok999wzTj755LSGBAAAYBPhe1YBAADInNTeWV3Tdddd11hDAQAA0MQ1WlkFAGjqKqpq4uFhfQpaH4ANo6wCAGyg8kVfxPo+HiWND2UBaK6cswoAAEDmKKsAAABkjrIKAABA5iirAAAAZI6yCgAAQOb4NGAAAFhFVVVtDDx+64LWBwqnrAIAwCoWLVq23uW+oggah8OAAQAAyBxlFQAAgMxxGDAAsMmqqKqOR4cdWtD6AGwcyioAsMkqX7Q81ndmoXMPgayqqayOTucfUtD6TZ2yCgAAkDELFi9f7/Lm8Mc256wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOb5nFQCAdaqsqonLh3YuaH1g46uprI6O5w0oaP3GpqwCALBOixd9sd7lZWVtYu7c8kaaDZCvBYuXr3d5FvdlhwEDAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5Pg0YABpARVVVTBx2WkHrAwD/R1kFgAZQvqgiyqNincuz+JUAAJBlDgMGAAAgc5RVAAAAMsdhwABAXiqqquPxY48uaH0AWBdlFQDIS/mi5bG+s3CdpwtAIRwGDAAAQOYoqwAAAGSOsgoAAEDmKKsAAABkjrIKAABA5iirAAAAZI6yCgAAQOb4nlWgyausqojLj3+moPUBAMiW1Mvq3XffHffee2+0aNEiSkpKYsKECWkPCTQzixdVRUTVOpeXlbWJuXPLG29CABuooqomHhy2Z0HrA2yqUi2rzz77bDz99NPx0EMPRevWrWPu3LlpDgcA0KSUL/oi1venNH9sA5qzVM9ZHTduXJx33nnRunXriIgoKytLczgAAAA2EamW1Q8//DDefvvtGDFiRBxzzDHx5z//Oc3hAAAA2EQUdBjw0KFDY8aMGWtd9tprr0VNTU3MnDkz/vSnP8XChQvjhBNOiJ133jl69+6d0zgdOrQuZJp1ysraNEhOmpnNLS+NzKznpZG5sfPS2EYNPWZjz9H9Jnt5+WRm/TbZ9xpmvKz/Xja133N1TRJnH9Mx1ymttn4W7odZv283hdvc3PLSyMx63oZYX9crqKw++uij613euXPnGDx4cBQXF0eHDh3igAMOiGnTpuVcVufPX9oghXXVcz4a6hfR0JnNLS+NzKznpZGZZt5XyfV8qjS24YaM2dhzbO73mzTuh1m/bzeFvKxvw8ae38oxN/bvpSHzmsJjbFPPyycz6/vexshsbqLboCcAACAASURBVHlpZG7svIZ6vFlf10v1A5YGDx4cr7zySvTu3Tu++OKLmDp1anznO99Jc0iAZqGiqjImHX1HQesDAGRZqmX1e9/7Xlx++eVxxBFHRETE0UcfHd/61rfSHBKgWShftCLKY8U6l/sEUWieKqpq4o/H7FhwBkAWpFpWN99887jhhhvSHAIAgP/nq74KJ8Ifs4CmI9VPAwYAAIB8KKsAAABkTqqHAQMAQHNXXVUbB59SVtD60BwpqwAAkKKFi5atd7nziGHtHAYMAABA5iirAAAAZE6zPQy4prIqykaeXXAGkJvKqor48YnPFJwBAMCmrdmW1QWLKyJi3S94nTsA6Vi8qCoi1v+HHvsfAABNpqzWVFZF2TknF7Q+NAdVVRVx6veeLWh9AABYn5rK6uj4g6EFZ6xPkymr3gmFDbPoK965tK8AAFCoBYuXf+V1NuR1Z3Fx0bqX5TwrAAAASJmyCgAAQOYoqwAAAGSOsgoAAEDmNJkPWAJoqiqqVsRDRz9dcAYAQHOirAKkrHxRZZRH5TqX+4RmAID6HAYMAABA5iirAAAAZI6yCgAAQOYoqwAAAGSOD1gCICqqKuOpIb8oaH0AgIakrAIQ5YtWRHms++txfGIxANDYlFUAGlxFVVU8NfSSgtYHAJo3ZRWABle+qCLKo2Kdy71TCwB8FWUVAMiEiqrqeOzYAQWtD8CmQ1kFADKhfNHyWN/77d6RB2hefHUNAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5PmAJgCahoqoqnjzmhwWt39xVVFXHxGOHF7Q+ADQWZRWAJsF3txbOp+0C0JQ4DBgAAIDMUVYBAADIHGUVAACAzFFWAQAAyBxlFQAAgMzxacCQg6rKFTH49EkFZwAAAOunrEIOFi2ujIjKdS73tQ+bhhVVK+I3w54uaH0AAAqjrAKsYckif5QAANjYnLMKAABA5qT6zurHH38cV1xxRSxZsiQqKytj0KBBMWrUqDSHZD2qK1dE11GPFbQ+AABAY0i1rN5www3Rv3//OPnkk2PZsmUxePDgOOSQQ6J79+5pDss6LHS+JQAA0ESkehhwUVFRlJd/WX4qKiqiqKgo2rdvn+aQAAAAbAJSfWf1sssui5EjR8af/vSnWLJkSVx00UXRpUuXNIfcqGoqq6LTOZcUtD4AAAAFltWhQ4fGjBkz1rrstddeiwceeCCOPvroOPPMM2POnDlxyimnxJ577hl77713TuN06NB6g65XVtYmp9zGzvsqJS1bRFlZiwbNzPU2NPZtTmPMjZ3XFLbhxt5GGyOzueWlkZn1vDQym1teGplZz0sjM+t5aWQ2t7w0MrOel0Zmc8tLIzPreYVmFlRWH3300fUuv/vuu+P555+PiIiOHTtG3759Y/LkyTmX1fnzl0ZtbbLe6zT0+ZZpnL+Za2ZD3FlyHS+X67fbqmXsfc7j+UyrTnXliv93Lu3/zaFQq96GrG/DlesUKu05NmZeGpnNLS+NzKznpZHZ3PLSyMx6XhqZWc9LI7O55aWRmfW8NDKbW14amVnP29DM4uKidb45mephwF26dIlXXnklhgwZEkuXLo2pU6fGt7/97TSHpBF91Qc2RfjQJgAAID+pltVrr702fvGLX8S4ceOiuro6Bg0aFIccckiaQwIAALAJSLWs7rnnnnH//fenOQQAAACboFS/ugYAAADyoawCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOaUbewKwqurKFXHAWRMLWn9VVZUr4vAzn8w7r2qNPAAAoHEoq2TKwsWVEVG5zuVlZW1i7tzyDc5b1MB5AABA43AYMAAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJnje1YzrKayMjr/4LcFrQ8AANAUKasZtmDxiohYsc7lZWVtYu7c8sabEAAAQCNxGDAAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOaUbewLQ3FVVrojhpz1d0PoAALCpUVZhI1u0uDIiKte5vKysTcydW954EwIAgAxwGDAAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZo6wCAACQOcoqAAAAmaOsAgAAkDnKKgAAAJmjrAIAAJA5yioAAACZU3BZfeyxx+LII4+MPfbYI+65557Vli1fvjx+9KMfxXe+850YMGBAvPjii4UOBwAAQDNQWmjA7rvvHjfeeGPceeed9Zb94Q9/iC233DKee+65+OSTT+Kkk06KZ599NrbccstChwUAAGATVvA7q7vuumt8/etfj+Li+lGTJk2KESNGRETETjvtFHvuuWe8/PLLhQ4JAADAJi7Vc1ZnzJgR2223Xd3/t91225g1a1aaQwIAALAJKEqSJFnfFYYOHRozZsxY67LXXnstSkpKIiLikksuiT333DNOPvnkuuU9e/aMF154Idq3bx8REVdeeWXsuOOOcdpppzXU/AEAANgEfeU5q48++mje4Z07d45///vfdWV15syZ0adPn7zzAAAAaB5SPQx4wIAB8cADD0RExCeffBJ///vf46CDDkpzSAAAADYBX3kY8FeZOHFiXH/99bFkyZJo0aJFtGrVKsaNGxdf//rX44svvohLLrkk3n333SguLo4LL7ww+vXr11BzBwAAYBNVcFkFAACAhpbqYcAAAACQD2UVAACAzFFWAQAAyBxlFQAAgMxRVgEAAMgcZRUAAIDMUVYBcvDhhx9u7CkAAClYsmRJLFu2bGNPg1U02bI6ZcqUGDNmTJx77rlx7rnnxpgxY2LKlCkNOsbll1+e13oLFy6Mn/3sZ3H66afHvffeu9qyUaNG5ZX5r3/9K95///2IiPjkk0/irrvuitdeey2vrLW54oorClp/5dwiIqqqquKmm26KU089Na677rpYvnx5odOLZcuWxT/+8Y9YunRpXuu/+OKLUVVVVfA8VrVgwYK62zZr1qx45plnCioyVVVVceedd8ZRRx0VvXr1ir59+8ZJJ50Uf/nLXwqa54wZM2LcuHHxy1/+Mn75y1/GuHHj4n//938LylzTmWeemdd6Wd9Xli9fXu/f97///aioqMjrfv3qq6/W/VxeXh4XXnhh9OvXL0aNGhXz5s3La45r+vzzz+PZZ5+Njz/+OK/1G3pfqa2tjaeffjrefPPNiIh49tlnY8yYMXH//fdHbW1t3rmzZ8+Ou+66K375y1/G2LFj46GHHooVK1bkndcY+0nE6veBXEydOjWeeOKJmDt37mqXP/roo3nlzZgxI0aNGhU//OEPY+7cuXHVVVfFPvvsEyeccEKD3e6zzz67QXIiIl5//fW49dZbC3o8fOWVV2Lx4sUR8eUL0ssuuyyOPPLIuPDCC2PBggU5591zzz15rbc+f//732PGjBkRETFt2rQYN25cvPLKKwVlTps2LX7xi1/EueeeG+eff37cdNNN9e5HuWiM118R+b0Gq6ysjNtvvz0uv/zy+Otf/7rasjFjxuQ9l7RfgzXkvrJ48eKYMmVKzJ8/P++Mhx9+uO6+PWvWrDj11FNjn332iRNPPDE+++yznPMa+nnlww8/jLPOOiuuuOKKWLx4cYwcOTJ69uwZxx9/fF6vw5YsWRKjR4+OffbZJ/r06RO9evWKQw89NO6+++6C5zp9+vR4/vnn4/nnn4/p06cXnLc2Df1H9JWPk/kqLy+vy1i8eHE899xz8dFHH+WdV5QkSVLQjDaC2267LZ5++ukYMmRIbLvtthERMXPmzJgwYUL0798/fvCDHzTIOIceemi9B7sNcf7550eXLl2iR48ecd9998WWW24Zv/3tb6O0tDSGDBkSEyZMyCnv7rvvjvHjx0d1dXWcccYZ8dhjj8Vee+0Vb7zxRpxyyilx0kkn5ZR3/fXX17vswQcfjOHDh0dExEUXXZRTXkTE0KFD6140/eY3v4kPPvgghg8fHs8880y0aNEi5yeJK664In70ox9F+/btY+rUqTFq1Kho165dLFiwIG644YY48MADc8rbfffdY6uttoojjzwyhg0bFt26dctp/TXdf//9MXbs2GjTpk2MHj06rr766th1113jvffei/PPP79uW+bi0ksvjdLS0jj44INj0qRJsdNOO8U3vvGN+M///M8YNmxYzr/niC9/r7fcckv069dvtX3lhRdeiB/84Ac5zXN95WzAgAHx0ksv5Ty/rO8r3bp1i6Kioljbw2RRUVG8++67OeWtup9cffXVUVtbGyeeeGI8+eST8emnn8Zvf/vbnPIiIkaOHBnXXHNNtG/fPl544YUYPXp07LHHHvHPf/4zfvKTn8TRRx+dU15D7ytXX311TJ8+Paqrq+PAAw+Mv/3tb3H44YfH66+/Hrvsskv8/Oc/zznz8ccfjxtvvDG6desWb775Zuy///6xbNmyeP/99+OOO+6I3XbbLae8htxPvko+zyvjx4+P++67L3bZZZeYNm1ajB49Ovr37x8Rq9+ncvH9738/DjrooFi6dGlMmjQpBg8eHMOGDYtJkybF66+/HrfddltOeT/84Q/rXfbyyy/HwQcfHBERN910U055xx9/fDzwwAMRETFhwoS49dZbo1+/fvHf//3fMXTo0Dj99NNzyouIGDx4cDz++ONRXFwcP//5z2OLLbaIwYMHxyuvvBLvvPNO3HrrrTnlde/ePUpKSmL//fePYcOGxaGHHholJSU5z2ulW265JR555JEoKiqKM844I/7rv/4r+vbtG1OnTo1BgwbFOeeck3PmuHHj4rHHHovevXvHf//3f0efPn2iuLg4nn/++fj1r38d++23X055jfX6KyK/feWyyy6L5cuXR/fu3ePhhx+O/fffP372s59FRP77SkM/rzT0vjJmzJi6Yv/WW2/FueeeG9tss03MmDEjbrjhhjjooINyyov4cl+ZOHFiRET86Ec/ih49esRRRx0VL7/8ckyYMCHuuuuunPIa+nnlpJNOigEDBkR5eXlMmjQphgwZEkOGDIm//vWvMWHChJxL5jnnnBPdu3ePQw45JJ544olo165d9O3bN26++ebYa6+94vzzz895jtOmTYuf/vSnsdlmm622r6xYsSJuuOGG2HvvvXPOXJd89pXp06fHj3/845g9e3YceuihceWVV0b79u0jIv99JSLiqaeeissvvzyKi4vj6quvjttvvz06duwY7733Xvz85z+PAQMG5B6aNEHf+c53koqKinqXL1++POnXr19OWX379l3rvz59+iR77LFHXvM76qij6n6ura1NrrzyyuT0009PKioqkqOPPjrnvCOPPDJZtmxZMnfu3KR79+7JzJkzkyRJkvnz5+eVt/feeycXXHBBcvPNN9f969WrV93P+Vh1HkOGDEmWLl2aJEmSVFVVJUcccUTOeUceeWTdz6ecckry9ttvJ0mSJB999FEydOjQvOb37rvvJmPGjEn69OmTDB06NLnnnnuSxYsX55yVJEkyePDgZM6cOcnHH3+c9OzZM/noo4+SJEmSOXPmJIMHD84rc+DAgXU/V1VVJSNGjEiSJEkWLVqU9O/fP6/M7373u8n8+fPrXT5//vzkO9/5Tk5Zu+22W9KtW7dkt912q/u38v/dunXLa35Z31cuueSS5LLLLkvKy8vrLjvssMNyzllp1TkcddRRSWVlZd3/873frLoNjz/++OTTTz9NkiRJ5s2bt9p+lMscG3JfGThwYFJdXZ0sXbo06dmzZ922XLFiRV6PDUmSJIMGDaq7X3/22WfJyJEjkyRJktdeey058cQTc85ryP0kSZJk7Nixa/133XXXJfvss0/OeYMHD67bbh988EHSv3//5KGHHkqSJMnrfp0kq99vvvWtb622LJ/7zQEHHJBceumlySOPPJI88sgjycMPP5z06dOn7v+5WvV2HXfcccnnn3+eJEmSLFy4MO99ZdCgQXU/DxkyZLVlq26PXOY4f/78ZNy4ccngwYOTb33rW8nYsWOTDz74IO/5ffHFF8m8efOSHj16JHPmzEmSJEnKy8vz3lcGDhyYfPHFF0mSfHl/Pu2005IkSZJ33303r+fShnz9lSQN/xps1fvG8uXLk3POOSe59NJLk9ra2rz3lYZ+XmnofWXV+/Lpp5+evPrqq0mSJMk777yTHHvssTnnJUmy2muONfeVfG5zQz+vrLq/HnzwwQXPb83HlOOOOy5JkiSpqKhIvvvd7+Yxwy/nMWXKlHqXT548Oa/Hm3vuuWed//bbb7+c80444YTkxRdfTBYuXJjcfPPNyYABA5IZM2bUzT1fQ4YMSWbPnp189NFHSY8ePZJ//etfSZJ8+Vx9zDHH5JXZJA8DTpIkioqK6l1eXJz7zUmSJMaPHx8PPfRQvX8r/8KQq8rKyrqfi4qKYvTo0bHrrrvGWWedlddhasXFxbHFFlvE1ltvHdtvv31ss802ERHRvn37tW6Hr/LII4/E7NmzY8stt4wf/OAHcd5550WbNm3ivPPOi/POOy/nvIgvt+PKQyNLSkpiyy23jIiI0tLSKC0tzTlv1e20bNmy6N69e0RE7LzzznkdSlJUVBTdunWLn//85/HKK6/EmWeeGS+++GIccsghccEFF+ScV1paGmVlZbHTTjvFVlttFTvvvHNERJSVleX9l/WSkpK627106dK6n7faaqto0aJFXpm1tbVrvR+3a9dure8Wrk9ZWVm8+uqr8d5779X9e/fdd+O9996Ljh075jW/rO8r1157bfTr1y++973v1b1znE/OSpWVlfHhhx/GBx98EEVFRav9XvN5/IqIqKioqDuctra2NnbYYYeIiOjQoUPOv+OIdPaVkpKS2GKLLaJVq1bRunXriIho2bJl3re5pKSk7n69/fbbx8yZMyMiYv/994+FCxfmnNeQ+0nEl+/EbLbZZrHFFlus9m/LLbfM+/6zcrt17do1/uu//ivGjx8f999/f955q663xx57rHPZhnriiSfqTtcYMGBAHHPMMbHFFlvE0KFDY+jQoQXNr6qqKrp06RIREW3bts3rOSXiy9/ntGnTIuLLx7OVh0kuW7Ysr0PSi4qKon379nHaaafFE088EbfddlssW7YsRowYESNGjMg5r0WLFtGqVavo0KFDtGnTJsrKyiLiy999vre5pKQkWrVqFRERX/va1+puc7du3VZ7/N1QDfn6a2VeQ74Gq6mpqft58803j5tvvjmWL18eF154Yd6nHTT080pD7yurmjdvXhxwwAER8eW7mfn8jiMidthhh7pD7nfYYYf45JNPIiLyPny8oZ9XampqYsWKFbFgwYJYsmRJ3SHLy5cvz+u1Q1FRUSxatCgiIv7973/X3Vc222yzvPe95cuXx7777lvv8l69ekVFRUXOeddcc038/e9/j+nTp9f7l8/v+YsvvohDDz002rZtW/f6/9RTT43PP/+8oNc5EREdO3aMnXfeObbZZpv4xje+ERFfPlevun/mIr/fwEY2ZMiQGD58eAwZMiQ6d+4cEV+efzNhwoScD3nbc889Y+HChWs9JKFTp055zW/77bePyZMnR+/evesuu/jii+PGG2+MO++8M+e8VR9gf/KTn6y2LJ/itssuu8Qf//jHuPPOO+M//uM/4sorryz4jvnPf/4zevbsWfdENnv27OjUqVOsWLEiryeI/fffP6677rr44Q9/GH369ImnnnoqBg0aFK+++mq0bds257xVX3C2aNEiBg0aFIMGDYpZs2blfKjpyoyXXnopysvLo6ioKJ555pno379/TJkyJe8n7cGDB8dxxx0XvXr1ildffbXu0KJCzmU88MAD48wzz4zjjjtutX3lz3/+c3zrW9/KKatPnz7x/vvvR58+feotW/nHhFxlfV+JiDjssMOiR48eMWbMmJg0aVLeD7YRXxbLs846q+7+uHI/Wbp0ad73m4EDB8YFF1wQF1xwQfTr16/uvOeXX345tttuu5zzGnpf2XbbbeP666+PZcuWRdeuXeOaa66JI488Ml555ZXYeuutc86LiOjSpUvcfvvtcdBBB8WTTz5Z92RYU1OT1++nIfeTiIhdd901+vfvv9bnlQcffDDnvBYtWsTcuXPrykvHjh3jj3/8Y5x++unx6aef5pwX8eUL+aVLl0br1q1X29cWLVqU1x/c2rdvHzfddFM8/vjjcfLJJ8eFF15Y0PPKxx9/HMcee2wkSRKfffZZ3Vwj8t+Xf/azn8WoUaOiV69eUVZWFscdd1z07ds3pk2bltd592v+IaN79+7RvXv3uPTSS+O5557LOa9NmzZx7733xtKlS6Nt27Yxbty4GDJkSLz88sux+eab55wX8eUfIi6//PI48MAD45lnnql78bxixYq8tmNDvv6KaPjXYFtvvXW89957dXklJSXx61//Oi6++OLVPlsjFw39vNLQ+8rs2bPj+uuvjyRJYvHixVFTU1O3D+db0EePHh3nnXdejB8/PrbaaqsYPnx4fPOb34yZM2fmdS5xQz+vHHnkkTFw4MCorq6OUaNGxahRo2K33XaLqVOnxuGHH55z3qmnnhpHHXVU7LHHHjF9+vS47LLLIuLL118r7+e56tKlS9xxxx0xYsSIutetixYtivvuuy+vzK5du8b3v//96Nq1a71l+Zw/vWLFitXuK0cccUS0bNkyvve970V1dXXOeSut+hx86qmnrnNZLprkOasRX57g/9RTT9V9EEHnzp1jwIABOZ9/UVlZGSUlJQWdZ7KmRYsWRVFRUWy11Vb1ln3wwQfx9a9/Pae8Rx55JPr371/3buVKH330UTzwwANx6aWX5j3X9957L6644or4/PPP4/XXX887Z12WLFkSH330UfTo0SOn9SorK+P666+Pxx57LNq2bRuff/55lJaWRp8+feLKK6+M7bffPqe8iy++OMaOHZvTOuvz9ttvx+jRo6O4uDiuueaa+P3vfx/PP/98bLbZZnHjjTfW/WUzVyvfufzmN78Zffv2LXietbW18fjjj8ekSZPq7StHH3103gWpoTSlfSUi4umnn4433ngjRo8eXVDOmpYvXx7z5s3L+X4d8eWLgLvuuivGjx8fCxcujKqqqthyyy3jiCOOiB//+MfRrl27nPIael9ZsGBB3HHHHVFUVBTnnntuPP3003HPPfdEly5d4rLLLsvrNs+bNy+uvfba+Oc//xl77rlnXHTRRdG+fftYtGhRTJs2re7crw3V0PvJq6++GjvssMNab9vUqVPX+tf29XnhhReirKys3h+FFi5cGP/5n/+Z1+cMrOsdsgULFsS8efNi1113zTlzpdmzZ8cVV1wRb731Vrzxxht5Zfztb3/7/9u787ic8vd/4K9bCqnR8LHMpLF+RsrWl7Era6NEi5Ilsi9ZQiGN7CQJWSqMMbaEcSdSmNFYsoQvExMxlmQNLaOU1vP7o999vt3Kcp/zPrrvup6Ph8dMp3HNde773Oc+73Pe7+tS+tnU1BQ1a9bEq1evcPLkSUFr+IHiz1pkZCTu3buHwsJCfPvtt7CysuLXlKkiICBA0FOhD0lKSoK/vz+qVKmC+fPnY//+/di1axeMjIzg6+sLU1NTlWNmZ2djy5Yt/PfKxIkT+RsVT548EbR28MqVK2V+VlS9/gLYX4MlJSVBW1u71I06juNw9uxZWFhYqBxTyu8VFp+VTZs2Kf08bNgw1KlTBykpKVi3bh1WrVolOL8LFy4ofVbMzc35J/WqYP29AhRfvwLFswSePXuG48ePo2HDhrC0tBQU7/79+/jnn39gbGyMxo0bi84vNTUVAQEBiI6OVtrev39/eHh4qHyz9siRIzAxMSnzuujYsWMYMGCASvGWLl0KCwuLUp+JP//8E97e3oLHBCEhIXBxceFvLircv38fGzduFFSbQ2MHq4SdgoICvHr1StCXtdSys7ORnJzMnyhVvfD+ktLT01GrVq1yHwCSyisrKwsFBQWCZh8QQgghhD3FFGP6bhaGrqoJqlatqpYDVQDQ1dWFsbExTE1N1XqgChSvh9KkgarQFhplEdq65mMGDhzINB7L/QXY7zOL/dXT01P6MmTdK471a8g6npiYz549w7Vr10qt/VGXeEeOHEFISAj/NEFhy5YtguJJ0TaKdXsd1vssRUzW+5yXl4eUlJRS24VOYQXY5/jnn3/ya/ivXr2K5cuXC5reDhS3uPDz8+OXCvz8888YNGgQPD09+Qt8VUjVkol16xqWr6EU+bF+X1jnqCnHjYKBgQEMDAxUrrL+KWLbTkodj1VcerJKSCWlaqlzKVrX3Lt374O/Gz16NGJjY1WO+SFCSruz3ucvub+A8PZbmhpPaMwjR45g5cqVqFu3LrKysrB27VqYmZkBEFbCn3W81atX46+//oKJiQlOnDiBcePGYfTo0YLjAezbRrFuryPFPvv7++P69evMYrLe59jYWMyaNQscx+G7777DunXr0KhRI8HxpMhx/fr1OH/+PAoKCtC5c2f8/fff6NGjB86dO4eOHTuqfKNj5syZqF+/PrKzs/Ho0SM0a9YMdnZ2OHHiBF6/fl1mq72PYd2SCWDfuob1a8g6P4D9+8I6R004bt6/CQgAGzdu5N/f8m47KUUbS6niamSBJULI5/nQCZvjOGRmZqoUy8zMrFTPUcXPQotD2NjYwNDQsMyKq0LujrLcX4D9PrPeXwAfHTALqYrI+jVkHU+KmNu3b0dERATq16+PuLg4zJ49G8uWLUP37t0FVQNmHe/s2bMIDw+HtrY2pkyZAjc3N2RlZWHatGmC4gHAo0ePsGHDBgBAv379sHTpUkyaNEnwnX+5XA65XA49PT3cv38fU6dORVZWFgYPHqw2+3zmzBmmMVnv87p167B7924YGxsjPDwcY8aMQVBQEIyNjQXvM+scT506hcOHDyMnJwfdu3fH6dOnLKv1jAAAIABJREFUYWBgABcXFzg7O6s80Lp//z7Wr1+PwsJCdO3aFTt27ICWlhbatGmDQYMGqZzfy5cvMWrUKABAaGgoJk2aBAAYOXKk4CeXBw8eRGRkJLKzs9GnTx+cOHECDRo0QFpaGsaOHavyoIP1a8g6P4D9+8I6R004blasWAFzc3OlWYG5ubn4+++/BcULDQ1F3759S62n1dXVVYt4UsatMIPVyMhI2NjYMIm1atUqeHl5MYml4OLigj179jCLd/nyZUHFDL5UPKC4at7atWuZxmSJ9T5nZ2eL/pC/T+yxuHv3bowfP77M4hWqDrbq1q2LiIiIMtsJCClaAQCGhoYIDQ0ts+qjkJgs9xdgv8+s9xcAJk+ejB9++KHMC08h04BZv4as40kRk+M4/j3p1KkTtm3bhokTJ8LHx0ct4gHg2xzVqVMH27dvx5QpU5Cbmys4Xllto/z8/AS3jQJKt9cZO3Ys8vPz1WafpYjJcp8LCwv5gkf29vYwNDTElClTEBgYKGqfWeaoaEWlp6eH7777jl92oKurK6hIkqItiJaWFr755hs+hkwmE7SshnVLJuD/Wtfo6uoyaV3D+jVknZ8iR4Dd+yLFa8gyPymOmx07diAgIAAODg580ae4uDj4+voKiieXy7Fo0SKYmppi9OjRkMlkkMvlgltOso4nZdwKM1jdvn07s8Gq0IpsH8N67Zivr6/g9SZfIh5Q3HqAJdYDftb7PGLECOavodhjkWULDSla11haWuLp06dlDt769euncjzWLUNY7zPr/QWARo0aYcWKFWVWnhUyAGb9GrKOJ1XMzMxM6OvrAwCaN2+OX375BePHj8e///5b7vH09PSQnJzM99DV09PDtm3bMGnSJNy9e1dQfqzbRrFuryPFPrOOyXqfCwoKkJubi2rVqgEAOnbsiLVr18Ld3V3wDQTWORYVFfEzS1auXMlv5zhOULsLRX/xatWqKU09z87OFjRIYN2SCWDfuob1ayhFyzbW7wvrHDXhuOnUqRN27NiBVatWISoqCgsXLhR104l120kp2lhKFperIGxtbdUyloKTkxPTeKxzlGKfWce0s7NjGq8yvIaxsbFccnJymb+7evWqqNjqqLLtL8dx3Pbt27mbN2+W+buff/5Z5XisX0Mp3hPWMQ8cOMBdvny51Pbk5GRuxowZ5R7v2rVr3D///FNqe25uLrdnzx6V43Ecx6Wnp3MZGRll/q6s/9en/PHHH1x8fHyp7WlpaZyfn5/K8aTYZ9YxWe/z5s2budjY2FLb4+PjucGDB6scT4ocz549y2VnZ5fa/vDhQ0Hnm5cvX3IFBQWltj9//py7cOGCyvGKiorK3J6amsrduXNH5Xgcx3GHDh3isrKySm2/f/8+t3LlSpXjsX4NWefHcezfF9Y5asJxU9Lp06e5QYMGcV27dhUdi+M47vbt25yTkxPXuXNntYzHOm6FKbAkl8vh4ODAJNbdu3dF9Zj7EkJCQjB58mS1jQcU35FiWaZ7yJAhOHDgALN4rPd52rRppfqdiaUJxyIhhBBCCPmwzMxM3L17V+U+2x/Cuu2kVG0sWcStMINVQgghhBBCCCEVh+Y0hSSEEEIIIYQQUmnQYJUQQgghhBBCiNqhwer/J6Qq3qds3boV6enpzOOqs23btn3WNlJ+Ll++zCzW+1X9WHBxcWEaj+X+Auz3mfX+AsUVEVli/RqyjidFzMjISLWOt2rVKqbxAPbH4i+//MI0nhT7zDom632W4rPCOkd1P7al+J5S9/ONFMcN6/eFdY6acNwkJSUxjacJ380s4mrkYHX16tWf/KNqY3M3Nzf06tUL8+fPR0REBFJSUkTn+fLlSwwYMABz585FfHy86HhRUVHIysoCAAQGBmLcuHGCmwtLEU8R83O2CcHiZJ6Tk4N169bBw8MDQHFj6T/++ENUzIsXL/ItdV6/fi24ZU90dDR++OEHdO3aldlrVhahPb7Kwro9ESBNmyeWWO8z6/0FIKgJ/Mewfg1Zx5Mi5vbt29U6nia0WDt69CjTeFLsM+uYrPdZis8K6xzV/diW4ntK3c83Uhw3rN8X1jlqwnEza9YspvE04buZRVyNHKweOXKEby78oT8REREqxTx27BgOHDiAzp0749KlS3BycoK1tbWoPBcsWIBTp07BzMwMPj4+cHBwwKFDhwT3SwsODoaenh5u3LiB2NhY2NnZYfny5YLzYxnv/Pnz8PPzw8uXL5VuGixcuFBwfu9jcTJfvHgxCgsLkZiYCABo0KCBqAq+W7duxaZNm7Br1y4AxVXPvL29BcUKDAzEvn37sHXrVqxbtw5BQUG4e/cusrKy8PjxY8E5vo9lTTUp6rNpa2szjcc6R9bxWO8voP77LMVxo+45qns8oPJ99qSIqe7xpIhZ2eJJEVPd40kRs7LFkyKmusdjFbcqozy+qG7dumHatGkf/W9UvbjnOA7Pnz/Hs2fP8PTpUxgYGDApL12jRg04OzujTp06WLlyJbZu3YrAwEB4eXmpPBiuWrX47Tp//jycnJwwcOBAUdN5WMbT1tZGzZo1IZPJoKury2+vV68eJk6cKDjHklh8iO7evQs/Pz/ExsYCAGrWrKnUrFpVkZGROHToEJycnAAUD34VT6tVpaOjg+bNmwMA9u7dixUrViAiIgJeXl4ICwvDli1bBOdZUv/+/ZnEAYBff/2VWSwFlu2JALb7C7DfZ9b7CwANGzZkGo/1a8g6nhQxR40apdbxVq9ezTQewP5Y9PT0ZBpPin1mHZP1PkvxWWGdo7of21J8T6n7+UaK44b1+8I6R004bv7nf/6HaTxN+G5mEldUl9YKpH379tzQoUO58PBw7sWLF0xivnr1itu4cSPXq1cvbsaMGXyT+OTkZM7CwkLlePb29lxERAQ3YMAALjk5meM4jhswYIDg/FjHKygo4Hbv3i3473/KoUOHRMdwcnLiOI7jbG1tOY7juHfv3nE2NjbM4nEcxw0cOFBQrPHjx3M3b94UnAshhBBCCCEViUZOA35fcnIyfv31V1FrDydNmgRdXV38/PPP2LRpE6Kjo5GWliYqLzs7O+Tl5SE0NBSBgYH44YcfAABGRkZwcHBQOZ6Pjw+OHz8OJycnGBkZISkpCZ06dRKcH+t4WlpaiI6OFvz3P0XIa/a+Dh06ICQkBHl5eYiLi4O7uzt69+4tOF6DBg1w9epVyGQyFBUVISgoCP/9738FxfLz80O9evUE50IIIYQQQkhFIuM4iSYoS2j06NHw8vKCsbExXrx4gYEDB6Jdu3Z48uQJbG1tMXnyZMGx8/PzERkZiY0bN+LFixe4deuW4Fi5ubmoVq2a4L9fUmFhITZv3owZM2YwiSeVTZs2oXnz5pJNJRArPz8fP//8M2JiYsBxHHr37o2JEyfyU6JV9erVK8ybNw+XL1+GTCZDhw4dsGbNGtSpU4dx5oQQQgghhFQuGjlYtba25qulbt26FX///Tc2bNiAN2/eYMSIEYIq3Z04cQIXL17ExYsXUVhYiM6dO6NLly4YMGCA4Dxzc3Nx5MgRPH78GAUFBfz2uXPnCoo3cuRI7N69W3A+70tKSsL8+fORkpKCmJgYJCQkICYmBtOnTxccs3PnzsjIyED16tVRo0YNcBwHmUyGixcvMstbHeXk5KCoqAg1a9Ys71Qkt23bNkyYMOGT24SKjIyEjY0Nk1hSSE1NxePHj9GuXTvBMbZu3QonJyd8/fXXDDMjYt2/fx/NmjX75DZ1IEVe6v7ZIxWPi4sL9uzZA39/f8yZM6e809FYRUVFOHfuHCwsLMo7lS9CyuPmzZs3AICvvvqKaVwinEYWWCr5tPLatWvo27cvgOIDS0tLS1DMkydPonPnzhg/fjyz4iTu7u7Iz89HmzZtoKOjIzpez549sX37dtjZ2SkVMapRo4ageIsXL8aUKVMQEBAAAGjZsiXmzp0rarB66NAhwX/3SyhrAb6+vj7atWuHLl26qBxv1KhRcHR0hKWlpVoOVPfu3fvJ/0ZXVxf29vafHTMqKqrUwLSsbUJt375d9AVzTk4OQkJC8OTJEwQEBOD+/ft4+PAhf65Q1fDhw7FlyxZwHAc7Ozt89dVXMDc3x7x58wTFU7S16t69O0aMGIG2bdsKilPSvXv3Sm3T19dH/fr1RcdWSEhIgKmpqdrGExvT09MT4eHhn9z2uXbs2AFHR0fo6+tjzpw5uHnzJhYsWIDu3burFCcnJ6fUtgkTJiAqKgocxwn+Dngfi8/eqlWrMHXqVNSoUQOjRo3CrVu3sGTJEtja2qoUZ/DgwZDJZB/8/W+//SY4xwcPHiA4OLjUjWShMVNTU7Fnzx4kJycrxQsMDFQpzqfO12JaUs2YMaPU66n47nNwcECVKqqtCktNTYWvry+eP3+OvXv3IjExEdevX8ewYcNUjpOeno7Y2FhMnz69VCFFVY/t9PR0rFmzBs+fP0efPn2UXrPp06dj48aNKsX7kEmTJjEreqjwxx9/oEGDBmjVqpXKf7dKlSoIDg5mPljNzMzEtm3bcPv2baVOFooOCJ/rzJkzH/29qnmzPm7S0tKwZs0afimb4kGLlZUVPD09Ubt2bZXivS85ORnJyckoLCzktwl9r6KiomBubg49PT0EBgbixo0bmDVrlqDjRgpSXHdq5GBVW1sb//zzD+rUqYMrV65gwYIF/O+EtoVRDNiys7ORnZ2tNBgU6tGjR0zXcPr7+/P/lMlk/Ifp9u3bguJlZmbC3Nwca9euBVB8shPbusDQ0BBZWVl49OgR84tQFlJTU3H16lV+0HLq1Cm0b98e0dHRsLKywpQpU1SKN2bMGISHh8PX1xd9+vSBg4MD82pvYmzYsOGTa3KvXLnyWSeN8+fPIzY2lm9PpCC0+vGHsJjssXjxYtStW1epRZGHh4fgwWp2djb09fURERGBgQMHwtPTE7a2toIHqwsWLICHhwcOHz4MHx8fVK1aFSNGjICNjY3gpQMTJ07E8+fPoa+vD6D4812nTh3o6Ohg7dq1op4EKwQGBmLr1q2i40gVT2jMtLQ0pKWlITc3F/fv3+ePwczMTGRnZwvORS6XY8yYMbh06RLS0tKwcuVKLF++XOXBqpmZGX/OL6ldu3aivgPex+Kzd+HCBXh5eeH06dOoX78+1q1bh4kTJ6o8WFV8tk6fPo0HDx7A0dERQPFramJiIirH2bNno3///nBwcBB8g7skNzc3mJiYoEuXLqLiKfqcp6en4/Lly/wN1IsXL/I3toSqW7cu/v77b362WFRUFOrWrYvo6GgkJiYqXUd9jgULFsDc3ByhoaEAgKZNm2LOnDkqD1YtLS3Rs2dP5OXl8ecoMdc3ixYtQsOGDWFhYYF9+/bh4sWLWL9+PapWrSq4BZy7u3upbZcvX+a3q3pT4kN+//13JCQkoH79+oJa9bVq1Qo3btxAmzZtmOQDAN7e3mjWrBmSkpLg7u6OQ4cOCbq2+/nnnz/4O5lMpvLAjfVxM2fOHHTo0AExMTH8jKe0tDSEhYVhzpw5olonrl69GocPH0aTJk34m0JC9lkhODgY1tbWfNvJUaNGYfny5QgLC1MpTlk3sBTHspgn1iyvO3lfuqITCxcvXuQ6duzItW7dmlu6dCm/PTY2lpsyZYqgmMnJyZyTkxPXqlUrrnXr1pyzszNfIVeoCRMmcJmZmaJiSMnJyYnLy8vj7OzsOI7juBcvXnD29vaiYp4+fZozNzfnevXqxXEcx924cYObNGmS6FxZGT16NJeRkcH/nJGRwU2aNInLzMzkrKysBMdNS0vjdu/ezTk4OHA//vgji1SZGD16NJP/huM4Li4ujtu4cSPXrVs3buPGjfyfHTt2cI8fPxabKo9F1WfFMc2iSjPHcfyxMX/+fO7cuXOlYgtVWFjInThxgrOwsOAsLS25Hj16cMeOHRMUa/ny5dzvv//O//z7779zAQEB3KVLlzhHR0fRuVZkv/76K9erVy/O1NSU69WrF/9n0KBB3IEDBwTHVRwj69ev58LDwzmO+79jUxVeXl6ct7e30veJ4hzLEovPnuJztnr1av5YFvNZcXFx4YqKivifCwoKOBcXFyY5ssI6npubm9L1R3JyMjdz5kxRMYcPH87l5ubyP797944bM2YMl5ubK+i7T3GtUPK9HTRokKj8WCiZQ1FREbd48WJu7Nix3Lt37wQfh127duXmz5/PyeVyTi6Xc4cOHeI6derE/8xayWsUVdja2nImJiaclZUVN3jwYP6PGIpjW9E1ITc3l3N1dRUVkyVWx83HrtssLS1Fxe7Xrx+XnZ0tKkZJiu+QoKAgbv/+/UrbVKE4fkv+UTh16pTg/Fhedypo5JPVzp0748KFC3j79q3SnHIzMzO+T6WqFi5ciCFDhmDw4MEAiu/eLly4EDt27BCcp76+PgYPHowePXooTQMWumYVAB4+fIj79++jb9++ePv2LfLz82FgYCAo1vDhwzFt2jSkp6dj48aNOHz4MGbNmiU4N6D4jspvv/3GTwlt3bo1kpOTRcVkKSUlBbVq1eJ/rlWrFp4+fQo9PT1RU7UVd6c4jpOsqbIQn3P8fu4x3rFjR7Rv3x4GBgZwcXERm9oHsaj6/P4MgdzcXFHvS8eOHfHjjz+C4zgsXrwYb968UXnaXEmvX79GWFgY5HI5WrduDX9/f/zwww94/PgxRo4cqXIPZqD4Tv9PP/3E/9y3b19s27YNs2fPxrt37wTnWhm4urrC1dUVISEhogr0va969eoIDg7G0aNHsW/fPnAch/z8fJXj+Pr64s8//8To0aMxffp0WFhYfHSKrFAsPnt16tTBggULcP78eUycOBEFBQVKU99UlZKSgtzcXFSvXh0AkJeXh5cvX4rKsV27dkhMTISxsbGoOApt27bFnTt30KJFCybxnj59CiMjI/5nIyMjPHz4UFTM169fK50Xq1atiufPn0NHR0fQd9/7RQnfvHkj6hyrmDqomMkgdHZbXl4e/+8ymQyLFi2Cn58fJk6cKHjm3dGjR7FkyRIkJCTAw8MDNWrUwKZNm1R7MqSCktcoqvD29macCfhjQ1tbGxkZGahVqxZevHghKmZmZiYePnyo9H4oumWoitVxU61aNVy/fh1mZmZK269duyZ6Gd8333wjetZiSTKZDEeOHMGxY8cQHBwMAIK+Vz52/IrpksHyulNBIwerQHGblPcXP+vq6sLa2hqnT59WOV5aWho/zQgoXi+j6pz89zVp0gRNmjQRFaMkuVyOrVu3Ij8/H3379kVKSgqWLl0quHGxnZ0dGjZsiD///BM5OTnw8/NDhw4dROdZt25dpZ9ZrNdlpXnz5vDx8YGDgwNkMhnkcjkaN26MvLw8QYOPmJgYhIeH4+rVq+jTpw+8vb2ZvIasFBQUICwsDA8fPkTLli2VjnEhFO2JpByssvB+i6IdO3aIOvkuWrQIiYmJMDIygo6ODrKysrB8+XLB8ezs7ODg4IDQ0FA0aNCA3y60rRVQXGDj2rVr/DT069ev82sdxQysK4O8vDzo6OjA1dW1zPWhQteE+vr6IjQ0FHPnzkXdunWRnJyMgQMHCorVq1cvtGvXDsuWLUN0dLSoAaCUAgICcOTIETg6OqJWrVp48uQJxowZIzielZUVnJ2d+Rs4iiUbYty4cQNyuRxNmjRRmnYvdM3q0KFD4eLiggYNGjCJZ2BggM2bN8PJyQlAcS0IoTelFTp27MhPx1Zc7LZv3x5v374V9B1taWmJhQsX4u3bt5DL5QgNDeVv9gvx+PFjeHh44Pbt25DJZDAxMYG/v7/SoP1zGBkZ4cqVK0qDn3nz5mHdunWClxzUrl0bgYGBOHLkCFxcXDBnzhxJbhaJ1bFjRwDiB24lNW7cGBkZGRg4cCCcnZ2hr6+Pli1bCo4XFRUFPz8/vHnzBvXq1UNycjKMjY0F1wVgddwsWbIEc+fORbVq1WBoaAig+KZRbm5umbVOVOHl5YXJkyejW7duSp81odP6fXx8sG3bNmZtJ1ljfd0JaGg14I+xsLD45ELustjb2yMgIABNmzYFUPwE08PDA3K5nHWKgtnZ2WHv3r0YMWIEDh8+DACwsbFBZGSkqLgsT2yjRo3C2rVrMWHCBISHhyMuLg6bNm1iWsVYjKysLGzevBlxcXHgOA6dOnXii4G8efNG5UX0o0ePhr29PX788Uf+zr86+emnn3D//n20b98e58+fR+/evUW3P1L39kQAuxZFikFMWQMYQPgghmVbK4WrV6/Cw8MD1atXh0wmQ05ODgICAtCyZUucPHlSsqcAFYG9vT3Cw8NhbGxcam0oyzWhrBw/fhxxcXFYtGhReaeipLCwEFOnTkVISAjTuDExMbh8+TI4jkOXLl3Qs2dPUfEuX75c5nbFxb6qrK2tMXjwYJiYmCitWRUaLyUlBStWrEBcXByA4tlk3t7eooql5efnIywsjH8dO3XqhKFDh4p64nPkyBGlc6yq65JLGjNmDAYMGKA0uy0yMlLlJzAZGRmQyWRlPp28d++e4Nl3CikpKVi4cCH++usv/v1RF6wGbh9y9epVvtaJ0LXZgwYNwo4dOzBu3DgcPnwY58+fx8mTJ7FkyRJB8VgdNwo3b97E8+fPARQ/EW3VqpXoGxOzZs3CgwcP0KJFC6XXzdfXV+VYmtDGUorrzgo3WO3Zs6egJ6tnz57FvHnz+DtGiYmJWL16tcqFMN4XGxtbqoratGnTBMVycnLCwYMHYWdnxw9WbW1tERERIShecnIyPD09+SI0LE5sN27cwKJFi/DkyRMYGxsjKSkJwcHBalGlrLCwEL/99hucnZ3LO5UvxtraGocPH4aOjg4yMzPh6uoq+gZMZWpPVNYgRkwRBymrfQLFg+uHDx+C4zg0bdpUrWY1VEasKmlqktGjR2PHjh1q+eRJKorzRGVS1rTnpKQkNG7cWFC8sq5lxFzfpKen89NVGzRoUCnahLEeuEnBwcEBcrkcAwcO5NtMDh06VOXiQAqsj5uSLly4gK5du4qOY2lpiRMnTjA7J7JuY8maFNedGjkNuKwWDQoly8arwtzcHJGRkbhx4wY4jkO7du1El6pes2YNbt68iXv37qFPnz44deqUoPYoCgYGBnj48CF/wEdERChNIVTVokWLmK/TbdOmDXbt2oVr164BKF5HrC69qrS0tBAREcF0sKruF6PVqlXjByz6+vpM1tOqe3sigF2LIsUFqOKGjliKap8svf/097vvvgNQfHMmJyeHWVsTojpWlTS/VDsOFtq2bYupU6fCxsZGqZ2X0MqXrNvMAOzP2z169MDZs2dhbm4uOKeScnJysGXLFjx+/JhJ6y2guBL+7t27S72OQivZ2tra4scff4S/vz//HTNr1izBg/YqVargwYMHSrPbhDy9S05Oho+PD27duoV69eoBKG4VZmJigiVLlggeTAPFVVhV7RjwJUmxnI01HR0dcByHRo0aYffu3TA0NER6errgeKyOm7LGFfPnz8cvv/wCjuNEPZFv3LgxsrOzmbU3ZN3GkjUprjs1crA6ceLED/5OzJOEOnXqoFu3bvxaILEXemfOnEF4eDgcHBywdOlSTJ06VfBUB6D4g+Ph4YGHDx+id+/eqF69uqjpVlKd2PLz81FUVASZTCb45oFUunbtiuPHjzObwsrqYlQqKSkpSgO3938WUuxL3dsTAexbFLEiZNrPpzg7OyM8PLzM9ibqOIW1Mnn06BE2btyIU6dOwcbGBpaWlh/9/voQKdpxSEVxo3Lfvn38NjFtGli3mQHYn7cPHDiArVu3ombNmvzFuJjZJqxbbwHFNzWaNWsmur2OQosWLdC4cWOMGjUKISEhMDAwEHVROmvWLIwYMQItW7bkz1tC1grOnTsXw4cPx44dO/i1+kVFRTh69CjmzZuH/fv3qxwzIiICnTt3xpkzZ/jvDnV8ms5q4CYld3d3ZGVlwdPTE4sXL0ZmZqao5QysjhsbGxt8++23Sttev36NCRMmQCaT4dSpU4Jz1NPTg4ODA7Niq6zbWLImxXWnRg5WY2JimMc8efIkli9fjlevXgEAkzdfR0cHVatWhUwmQ35+PurXry+qilrTpk1x8OBBJCUlgeM4NGnSRNSJSIoT28mTJ+Hj44NWrVqhqKgIiYmJWLZsmagvWZb27NnDdAorq4tRqQwfPvyjPwtx5swZLFy4EFpaWoiJicHNmzexefNm5uvUxHj58iXkcjm/bsnNzQ3z5s1DaGgohgwZUm6DVdaN0QH2T38JO6wqaT569AgbNmwAAPTr1w9Lly7FpEmTEBQUxDRfFlhPTysqKmJaoRlgf95mPdvk7t278PPzQ2xsLACgZs2aKCoqEhXzzZs3WLZsGYv0ABRfP8yaNQsHDx7E8OHDERwcLGqao7m5OY4dO4b4+HhRs9syMjIwaNCgUrna2trylVNV9fjxYxw8eBB3796Fj48PTExM1LLCOquBm5Tq1asHfX196Ovr88VB79+/Lzgeq+Nm2rRpiI+Px+LFi/kCS71792Yy3mjatCl/nc2Cun/XS3HdqZGD1X/++QevXr0qNZf83Llz+OabbwQ9rvf398f69evRrl07ZpUza9asiZycHJiZmcHLywt169YVNRh0d3dHYGAgmjVrVmqbECVPbMD/rdMVY926dQgLC+OrICclJWHKlClqM1hlfVEhRVl3loSuj/4YdW9PBEjXokgs1o3R35eWlob4+HjIZDK0bdu2UqzTUmesKmlK0Y5DKhzH4bfffsOjR4/g6emJJ0+e4OXLl3yValWxbjMDsD9vs55twrr1FgD897//RUpKiqgiTSUp8nFycsI333yDsWPHfrAQ3eeqXbs2evXqJSqGgYEBIiMjMWDAAKWWckePHhW8JGnatGmYNm0aHB0dMWjQICQkJOD169fo378/WrZsiXXr1onKWSzFWmFWAzcpeXp6lnoiXdY2VbA4bqZNm4Zbt27Bw8Omv5m5AAAX+0lEQVQDtra2GDZsGLM1plJch7FsY8maFPurkYPVgIAAzJw5s9T2WrVqYc2aNYKe8NSqVUvwl+mHrF27FlpaWpg3bx527NiBzMxMwQNLAGUOCB48eCA4nhQntlq1aim162ncuLHafIAA8HfMWGFd1p21PXv28G1m3l9TtXHjRkyfPl1QXHVuTwSwb1HEipRFERSzGkxNTcFxnNrNaqiM1qxZA6C48Enr1q35SpqqkqIdh1R8fX2RmpqKhIQEeHp6ombNmli5cqXgNaas28wA7M/brGebsG69BRQ/WR00aBDMzMyUXkeh1yQl1013794dQUFBgtvosbRq1SosWrQIS5cu5QfmKSkpMDY2xqpVqwTFtLKywvfff8/f2Bg5ciQiIiIQHh6OhIQElukLMnv2bMjlcri6umLnzp2iB25SSEtLQ1paGnJzc3H//n3+ZkdmZibfkaK8mZiYYNeuXdiwYQNcXV0F9S4tC+u1zqzbWLImxXWnRlYD/li7lpIVxj6H4k7grl27oK+vD2tra6UTuTosWD5w4AD279+PBw8eKD1VzczMRJMmTVT+QvzU3U8h+6yIuX37dmhpacHR0REcx0Eul0NHRwdjx45VOaYUnj9/Dn9/fyQmJio9kRCzHkGBRVl31kquq3l/jY3QNTfq3p4IYN+iSAosG6MDxRdUQUFBpWY1REdHi86ViJOXl6fUF1XVc6zU7ThYsrW1xeHDh2Fvb89XrVf1e7kk1m1m3sfivD148GCEhIRgwoQJ/D5bW1sjKipKUDxWrbdK+tC5vqK2tEpLS1NqQSL2nH/79m1MnToVJiYmuHv3Ll6/fo3x48eja9euaNeuHYuUBbO2toa7uzvWrFmDBQsWlPq92Bk7LOzcuRM7d+7Ey5cv+cJXQHEBHhcXF76nsLq4fv06rly5Imp5gGKts7u7O1/tmMVaZ6naWLIixXWnRj5Z/djdDlXvhLxflGTp0qXMFiw/ePAAISEhSE5OFlXFsFu3bmjUqBGWLVumtDBZT0+vVOn4z6HY5w8Rss/vv44l79bKZDK1Gax6e3vD2toat2/fxpo1a7Bv3z6+eqoYeXl5/PSvvLw8tbjJAUBp6tj796WE3qfy9PTEhAkT8OTJE4wcOZJvT6QuCgsLcezYMcybN6/M36vDQJV1Y3RA/Wc1VEasaiF87H1Up4EqUFwJsuT3i9i1lqwGpR/SoUMHJnFYzjbR1tbGlClTmD6NYTUonTNnDvz9/TF48OAyryPEPPFmqXbt2kzP9S1btkTdunWxadMmAMWDA0NDQ4SFhZX7YHX27NnYv38/Xr9+XWqpCYvlJSy4urrC1dUVISEhzNegS8HMzEz0OlOp1jpra2uXqiysLg9IAGmuOzVysFq7dm3cunULJiYmSttv3bql8sWZlAuV3d3dYWtrC3t7e1EHkqGhIQwNDfm7JmlpaaJOwop9Dg4Ohra2NpydncFxHA4ePCi4Qbi6L/hWSE9Ph5OTE3bt2gUzMzO0bdsWo0ePFhxPisJcLJW8mHj/wkLoegx1bk8ESNOiiLWQkBDI5fJSjdGFUMxq6N69O4KDg5VmNfTr149l2kRFUtRCUHfff/89jhw5Ao7j8OTJE2zduhXt27cXHE/d24MBxfUpXr9+zZ9T4+LioK+vLzheVlYWgoKCcOnSJchkMnTq1Alubm7Q09NTOdbOnTvh6ur6wXoUqlbmdHV1BYAP3gxUZ2Ke8APFN7sV6tatC1tbW9ja2rJITZS+ffuib9++8PX1xfz588s7nY+aPHky7t27h7i4OADFfdtLzhgsL4mJifD29kaVKlXg5+cHPz8/xMXFwcDAACEhIYKWCUi11pl1G0vWpLju1MjBqpubG9zc3DB16lS0bt0aAHDz5k0EBQWJag3DWtWqVTF+/Hhm8W7cuAF3d3cUFRXhzJkzuHnzJg4cOCC4wt/Zs2eV2guMGzcOw4YNw6hRo1ilrHYUg3FdXV08e/YM//nPf/Ds2TPB8dT9YrRkyfCS/85xHF6+fCk4rjq3JwLYtyhirWrVqqhTpw4/NbRbt26Ce2VqyqyGykiKWgjqzsvLC6tWrcKrV68wZMgQ9O7dG15eXoLjqXt7MID9bBNvb2/o6elhwYIF4DgO4eHh8Pb25itCq0KxrKlkP0YxWrVqBUD5iXdeXh7+/fffUk+Xy0NZ/TIVxPTzBIp7CCuI6UcvFXUfqALA4cOHsWbNGvTs2RMAsGXLFnh6epaq4PylLV++HFOnTkVmZibGjx+PWbNmYevWrYiJiYGfn5+g9aBSrXVm3caSNSmuOzVysNqjRw8sX74cQUFBWLFiBQDA1NQUS5cuRY8ePco5u//DulH4ypUrsW3bNnh6egIorsIq5iIgIyMDjx49QqNGjQAUF3DKyMhgkqu66tChAzIyMjBs2DA4ODhAR0dH1IBG3S9GS5YMf798+LBhwwTFVPf2RAD7FkWssWyMrimzGioTxdPufv36ITQ0VC1rIUhFT08Py5cvZxZP3duDAexnmzx48EBp/Vn79u1hY2MjKNbQoUMBsK/QOWvWLCxduhTa2tqwtbVFeno6Jk2ahHHjxjH9/6hKMT23rOmGYq9vzpw5oxZTajXZL7/8gvDwcP7GxqtXrzBu3LhyH6y+ffsWffr0AVB8w1eRT+/evQXdJAKA6Ohofq3zhg0b+LXOQUFBpbqZqIJ1G0vWpLju1MjBKlA85a179+7lncZHdenSBW5ubqhSpQqTRuH5+fml1icJnbYLFH/ZDBkyhL9TeuvWLaZ92NSRYuqSnZ0dOnbsiKysLHz//fcqx9GUi1EpSoire3sigH2LItZYN0Yn6kXKWgjqrn///nBxcYG9vX2pdVVCqHt7MAV9fX1mA5lvv/1WablPeno6GjZsKCjWp9rRqToNWOHhw4fQ19fH8ePH0alTJ8yfPx9Dhgwp98GqoaEhQkNDy2zRI/T9efHiBRo0aIDg4GA+ho+PT4W/XpJKySfw6vA0HlBeS9mtWzel34lZdy/FWmfWbSxZo9Y1/9/FixfRpUsX0f+N1BYuXAhfX1+YmpoymSKqo6ODt2/f8nO+7927pzRAUpWlpSU6dOiAv/76CxzHwczMTC2Kz0ihrArIX3/9Nb7++mvk5OSoPLjUlIvRe/fuKRWaiIqKQmRkJIyMjDB9+nRBa6A0oZAP6xZFrCnOTSUbo5OKozI/7V67di327t2LzZs3w9LSEsOHDxdUCFBB3duDsaQYWOrq6sLW1pZvQXL69GnB1zOK6b/Jycm4cuUKv479jz/+EHXDX7H848qVK7CwsECNGjXUYimMpaUlnj59WuZgVega/nnz5iEtLQ3p6ek4evQoTE1NcfPmTbGpVkrfffcdNmzYAGdnZ8hkMhw4cABGRkblnRbfK/n9mSEvXrwQ/fCB9Vpn1m0sWZPiulMjW9fY2tpi//79H60qNWzYML6kc3lxdHRkWhnvzJkzCA4OxuPHj9GjRw+cO3cO/v7+oqYTVBbGxsZKg0sAajm4ZM3R0RFBQUGoV68ebt68CVdXV0yZMgV37txB1apVVeo7pyntiQBpWxSxkJqaCl9fXzx//hx79+5FYmIirl+/LniKDCHq5s2bNzh06BB+/fVXNGzYEK6urrC0tBQVUx3bg7GkePryIWKeWEyYMAGrV6/G119/DaD4aa2Xlxe2bNkiKJ67uzsyMzPx4MEDREVFoUqVKnB2dkZERITgHNXZu3fv4ODgAGtrayQkJODChQuwsLBAly5d6LytgtTUVCxfvhwXLlyATCZD165d8dNPP6FOnTrlnVqZsrOzkZOToxb5sW5jKRWW150KGvlk9c6dOzAzMytzsKoYgPznP/8ph8yU9e3bF/v27YOVlRWTKaIWFhZo2rQpzp07B47jMGXKFH69Kfm4yvqkIzc3l+9pduLECdjb22PChAkoLCxU+c6eJhXykapFESsLFiyAubk5QkNDARSvQZkzZw5d9JAKIz4+HnFxcahevTp69OiBsLAwREVFYf369YJjsmozo66kmD6n8Pz5c36gChTPLHr69KngeH5+foiNjUWLFi2gq6uLlJQUeHh4sEhV7cyYMQOdOnWCjo4O/x7Z2trCzc1NbeogaIpq1aqVqoKblZVVTtl8mq6urujiZKzWOrNuYykVltedCho5WNWUgYfiS3nJkiXMnuIZGRmVWrBMyIeUnJYVHx+PESNGAChu76LqlC1N+dwB7FsUsZaSkoJhw4Zh//79AIqn+KvDFDpCxPrll18QFhYGIyMjjBw5EhYWFpDJZJg8eTK1UvpMwcHBTHusAsU3xH766Sc4OjoCAORyuag+ktWrV1eqU1C/fv0yp95WBFOnTsWlS5fw7NkzWFpawsjICKmpqXj79m2F7p4ghZEjR5bqJ17WtoqA9Vpn1m0spcLyulNBIwermoL1xf3//u//IiAgAMnJySgsLFS7CqdE/XzzzTfYu3cv6tevj4SEBH7dU15enlq2nGGFdYsi1qpWVT71vnnzRnCzbELUyePHjxEcHFxm70ShfQUri4iICHTu3BlnzpzhB6v29vZMLuRXrlyJzZs3Y9myZeA4Dp07d9bIXqnloUWLFmjRogWioqKwf/9+JCUlYezYsZDL5Vi0aJFS5WZStoKCAr7l3bt37/jvu8zMzDJrilQEUq11Zt3GkjUprjtpsKpBvL29MXPmTLRq1YqewpDPsnDhQixZsgQpKSlYsmQJatWqBaC4AJmiz1lFxLpFEWuWlpZYuHAh3r59C7lcjtDQUAwePLi80yJEtI9VtVZUnidle/z4MQ4ePIi7d+/Cx8cHJiYmePfuHZPYenp6NDgVSTGFsXHjxvjqq6/41onk00JCQrBp0ybIZDKlKrh6enoYM2ZMOWYmnZ07d/JrnR89eoSoqCg8fPgQM2bMELXWmXUbS9akuO7UyAJLlZWTkxMOHjxY3mkQolGePXsmuEWRlI4cOYKYmBhwHIfevXuLqg5IiLpQ9+JmmsDR0RHz5s1DQkICNm/ejDp16qBly5ainkxLMbW4Mnv69KnaV51XR0uXLsXChQvLO40vQrHW+eDBg3zBV1tbW/j5+eHixYuCB+mDBw/GoUOHYGdnx8ct+e8VET2e0yA2NjbYt28fMjIykJOTw/8h5EM+Z4p4RZpGXvJzofjz9ddfw8jISG0+K4WFhdiwYQMGDRqE9evXIzAwkAaqpMLw9vbmp32tWbMG7du3h729fTlnpRmsrKzg7u7O95MdOXIkGjZsiOPHj2P8+PGCYkZERCAlJQVnzpzht9H7IR4NVIWpLANVoHitc0FBAb/Wedy4cUzWOrNuY8maFNed9GRVg0RGRsLHx4efFlTR264Q8TSlzRMrmtKiaOTIkdi9e3d5p0EIc4o7/AMHDsTRo0dRVFSE0aNHY9euXeWdmka4ffs2pk6dChMTE9y9exevX7/G+PHj0bVrV6Xpk59r06ZNuHTpEm7duoUBAwbAxMQEu3btQnR0tATZE0Le5+zsrLTWuUuXLoiPjxe81lnd21hKcd1Ja1Y1yNq1a7Fr1y6YmprSmlXyWTSlzRMrmlKxuGfPnti+fTvs7OyUyuKLbT5OSHlT9+Jm6q5ly5aoW7cu33PVxsYGhoaGCAsLEzRYnTZtGqZNmwZHR0cMGjQICQkJeP36Nfr37y96ajEh5NNYr3VW9zaWUlx30pNVDTJ06FCEhYWVdxqEEJGMjY35f1fHJ7+ECOXn54dJkybh9OnTWLVqFV/czNvbu7xT0xjx8fFo27YtAGDMmDHYsWOH4FhWVlb4/vvvkZCQgKCgIDRr1gyOjo4IDw9HQkICTE1NWaVNCPkEWussDA1WNcj69euRn58Pa2trpfnpzZs3L8esCCGEkNLUtbhZZcN6ajEhpPxUxjaWNFjVIL179y61TSaTUZVFQggh5epTBcxoivvnO3PmDCwsLJjGVKybA4qnFk+YMAEXL17EqlWrmP5/CCHS+vHHH8tsY1mRn9jSmlUNEhMTU94pEEIIIaWYmZlpRHEzdfbixQs0aNAAwcHB/GDVx8cHy5YtEx275DTsunXrwtbWlqqQE6KBvvrqK1hZWZV3Gl8UPVnVQKmpqUr967799ttyzIYQQgghYrm6uiItLQ3p6emYN28eTE1NMXv27ApTrZ0QIt7OnTuho6MDKysrpSWBFXn2Cg1WNcjFixfh5eWF1NRUVKlSBfn5+TAwMKjQ89QJIYSQyuLdu3dwcHCAtbU1EhIScOHCBVhYWKBLly4YNmyY4LhSTC0mhHx5lbGNJfU/0SD+/v749ddf0bx5c8THx2PJkiVwdnYu77QIIYQQItKMGTNw6NAh6OjoYNq0aQgODkbjxo3h5ubGX5iq6sWLFwCA4OBgfpuPjw+TfAkhX56ijWVCQgJu376NxMTECj1QBWjNqsZp0qQJCgoKIJPJMGTIEIwYMaK8UyKEEEKISFOnTsWlS5fw7NkzWFpawsjICKmpqXj79i1GjRolKOa8efP4qcVHjx6Fqakpbt68yThzQsiXUq9ePbRu3bq80/ii6MmqBqlatfjeQv369RETE4M7d+7wd00JIYQQorlatGgBV1dXNGnSBCdPnoSPjw90dHQgl8sFF0PauXMnDh48iK+++gqPHj2Cv78/Hj58iBkzZmDfvn2M94AQIrXOnTvD398fCQkJuHfvHv+nIqMnqxpk1KhR+Pfff+Hu7g4PDw9kZmZSo3VCCCGkAlEMTBs3boyvvvoKK1asEBxrxowZ6NSpEz+1WBHfzc2N6l0QooGOHDkCAIiOjua3VfQ2llRgSUMUFRXh3LlzVCCBEEIIqSSePn0qqn/inTt3cOnSJWzevBkGBgYwMjLCnTt3EBgYiHbt2kFLS4thtoQQwh4NVjXI0KFDERYWVt5pEEIIIUSDODs7Y//+/UhKSsLYsWPRpUsXxMfHIzIysrxTI4QIUJnaWNI0YA3SqlUr3LhxA23atCnvVAghhBCiIVhOLSaElJ/K2MaSnqxqEDs7O/zzzz9o1KgRdHV1+e2//fZbOWZFCCGEEE0hdmoxIaT8ODg4ICAgALNmzUJ4eDgOHjyIZ8+eYebMmeWdmmToyaoGoWJKhBBCCBGDBqqEaLbK1saSBqsapGPHjuWdAiGEEEIIIaQcvN/G0tDQsMK3saTBqgbJzMzEtm3bcPv2baVF1bt27SrHrAghhBBCCCFSq4xtLKuUdwLk83l7e6NKlSpISkrCkCFDoKWlRcWWCCGEEEIIqeCKioqgr6+PWrVqoU2bNvj9999x6dIlDBo0qLxTkxQNVjXIo0ePMHPmTFSvXh02NjbYsmUL/v777/JOixBCCCGEECKhKlWqIDg4uLzT+OJosKpBdHR0AADa2trIyMiAtrZ2hZ+nTgghhBBCCPm/NpaVCa1Z1SCNGzdGRkYGBg4cCGdnZ+jr66Nly5blnRYhhBBCCCFEYlevXsW+ffsqVRtL6rOqoa5evYrMzEyYm5tDS0urvNMhhBBCCCGESOjy5ctlbq/IHUNosKph0tLSEB8fD5lMhjZt2qB27drlnRIhhBBCCCGEMEfTgDXIyZMn4ePjg1atWqGoqAiJiYlYtmwZ+vbtW96pEUIIIYQQQiRUGdtY0mBVg6xbtw5hYWFo0qQJACApKQlTpkyhwSohhBBCCCEVnLe3N5o1a4akpCS4u7vj0KFDMDU1Le+0JEXVgDVIrVq1+IEqUFxwycDAoBwzIoQQQgghhHwJlbGNJQ1WNUj37t0RHByMV69e4eXLlwgJCUG/fv2Qk5ODnJyc8k6PEEIIIYQQIpHK2MaSpgFrkE2bNgEAAgMDlbavXr0aMpkMt2/fLo+0CCGEEEIIIRKrjG0sqRowIYQQQgghhGiQytLGkp6sEkIIIYQQQogGKNnGsm3bthV6oArQmlVCCCGEEEIIUXsnT56ElZUV9uzZg507d2LAgAH4448/yjstSdE0YEIIIYQQQghRc1ZWVggKCirVxjI6OrqcM5MOPVklhBBCCCGEEDVXGdtY0pNVQgghhBBCCFFzmzZtgpaWFhwdHcFxHORyOXR0dDBs2DAAQI0aNco5Q/ZosEoIIYQQQgghas7Y2PiDv6uobSxpsEoIIYQQQgghRO3QmlVCCCGEEEIIIWqHBquEEEIIIYQQQtQODVYJIYQQQgghhKgdGqwSQgghhBBCCFE7NFglhBBCCCGEEKJ2aLBKCCGEEEIIIUTt/D/uuIqZ5SIH8QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 可视化\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "sns.set(style='darkgrid')\n",
    "plt.rcParams['figure.figsize'] = (16, 8)\n",
    "\n",
    "# 权重分布\n",
    "s_scores = start_scores.detach().numpy().flatten()\n",
    "e_scores = end_scores.detach().numpy().flatten()\n",
    "\n",
    "# 对应的标签\n",
    "token_labels = []\n",
    "for (i, token) in enumerate(tokens):\n",
    "    token_labels.append(\"{:} - {:}\".format(token, i))\n",
    "\n",
    "# 可视化权重分布\n",
    "ax = sns.barplot(x=token_labels, y=s_scores, ci=None)\n",
    "ax.set_xticklabels(ax.get_xticklabels(), rotation=90, ha='center')\n",
    "ax.grid(True)\n",
    "plt.title(\"Start Word Scores\")\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-06T07:39:37.389249Z",
     "start_time": "2020-05-06T07:39:37.039885Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function matplotlib.pyplot.show(*args, **kw)>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6QAAAI0CAYAAAAKruQqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdeZgU9b0+7M8s4AYBwQE1uHJU1IigIGpcowmLqCCiaPRoxLhFNAlxTRSVREVjjD839OSAiRqTuCGiqNEYNepR4aiEGBPFNWFfHZBhtnr/4GUOMGy9Uc3MfV/XXNdMV9fT367u6upnqqq7JEmSJAAAAGAjK017AAAAADRPCikAAACpUEgBAABIhUIKAABAKhRSAAAAUqGQAgAAkAqFFACKzL/+9a/YY489ora2Nu2hAEBBKaQAFLVJkybFkCFDYv/9948DDjgghgwZElOmTImIiMceeyxOOeWUnPLXV/5mz54de+yxR8ydO7fhsrvvvnuNlw0dOjSnsWyodS0TANiUKKQAFK3FixfHeeedF6eddlq8+eab8fLLL8eFF14YLVu2zEv+huyB7NChQ+y0007x1ltvNVw2adKk2HXXXRtd1rNnz4KMYWWFXiYr1NXV5TUPANZEIQWgaH388ccREdG/f/8oKyuLzTffPA455JDo0qVLTJs2LUaMGBHvvPNOdO/ePXr06BEREX/+859jwIABsd9++8Xhhx8et99+e0Peir2hDz/8cBxxxBFxxhlnxGmnnRYRET179ozu3bvH22+/3WgcPXr0iEmTJkXE8qL23nvvxX/+53+uctnbb7/dMIbKysq49NJL48ADD4wjjzwy7rrrrqivr4+I5Xt1hwwZEtdff30ccMABcfvtt0ddXV2MGjUqevXqFUcddVS89NJLWS2TFf7whz9E3759o3v37tGvX7/429/+FhER06ZNi9NPPz169OgRxxxzTLzwwgsN81x++eUxYsSI+O53vxvdunWLN954I6qrq2PUqFFxxBFHxMEHHxxXX311VFVVRUTE/Pnz49xzz40ePXrEAQccEKeeemrDfQSADVWe9gAAYG122WWXKCsri8suuyz69esX3bp1izZt2kREROfOnePaa6+Nhx9+OB566KGGebbYYosYNWpU7LbbbvHPf/4zzjrrrNhzzz3j6KOPbrjOW2+9FU8//XSUlpbG3Llz46ijjoq33norysvXvFns2bNnjB07NiIi3nvvvdh1113joIMOarjd9957L2pra6Nr164RETFy5MiorKyM559/PhYuXBhDhw6NioqKGDx4cERETJkyJY455ph47bXXora2Nv7whz/Eiy++GOPGjYstttgihg0bltUyiYiYOHFi3H777XHnnXfGPvvsE5999lmUl5dHTU1NnHfeeTFo0KD47//+75g8eXJccMEF8eijj8auu+4aERETJkyIe++9N+65556oqamJm2++OT7//PMYN25clJeXx49+9KO48847Y/jw4TF27Njo2LFjvP766xER8e6770ZJSUlmDzAAzZ49pAAUrVatWsVvf/vbKCkpiauuuioOOuigOO+881Y5d3N1vXr1ij322CNKS0ujS5cuccwxx8Sbb765ynWGDRsWW265ZWy++eYbNI6ePXvGBx98EIsWLYrJkydHjx49Yuedd44FCxY0XLbvvvtGy5Yto66uLp5++ukYPnx4tGrVKjp16hTf+c53Yvz48Q15HTp0iNNPPz3Ky8tj8803j4kTJ8YZZ5wR2223XbRt2zbOPffcrJfJI488EmeffXZ07do1SkpKYqeddoqvfvWr8e6778aXX34Z55xzTrRs2TIOOuigOPLII+Opp55qyD7qqKNi//33j9LS0mjZsmU8/PDDceWVV0bbtm2jVatWce655zZcv7y8PObMmRPTp0+PFi1aRI8ePRRSADKmkAJQ1Dp37hw33nhjvPzyy/Hkk0/G7Nmz4/rrr1/r9d999904/fTT48ADD4z9998/fve738WCBQtWuc62226b0Rg6deoU2267bUyePDneeuut2H///SMionv37g2XrTh/dMGCBVFTUxPbb799w/zbb799zJo1a623P3v27Nhuu+1Wuf66rGuZzJgxI3bcccdG88yePTu23XbbKC39v03/6uNaeQzz58+PpUuXxgknnBA9evSIHj16xNlnn92wLIcOHRo77bRTnHXWWXHUUUfFvffeu84xA8CaKKQAbDI6d+4cJ5xwQnzwwQcREWvcIzd8+PCG8zAnT54cQ4YMiSRJVrnOyvNt6F69/fffP95666145513Yr/99lvlssmTJzcU0q233jpatGgR06dPb5h3xowZ0bFjx7XeZkVFRcyYMWOV62+o1ZfJdtttF5999lmj63Xo0CFmzpy5ynmeq49rZVtvvXVsvvnm8dRTT8WkSZNi0qRJMXny5IZzbFu1ahWXX355vPDCCzF69OgYO3Zsw+G7ALChFFIAita0adNizJgxMXPmzIhYXqAmTJgQ++67b0REtG/fPmbNmhXV1dUN8yxZsiTatGkTm222WUyZMiUmTJiwztto165dlJaWxueff77O6/Xs2TOeeOKJ6NChQ7Rq1SoilhfSJ554IhYvXhzdunWLiIiysrLo06dP3HrrrbF48eL497//HWPHjo3jjjturdl9+/aN+++/P2bOnBmLFi1a597G9S2TE088McaMGRNTp06NJEni008/jX//+9/RtWvX2GKLLeJXv/pV1NTUxBtvvBF/+tOfol+/fmu8ndLS0hg8eHBcf/31MW/evIiImDVrVrzyyisREfHiiy/Gp59+GkmSRKtWraKsrGyVva8AsCF8qBEARatVq1bx7rvvxtixY6OysjJat24dRx55ZFx66aUREXHggQfGf/zHf8QhhxwSJSUl8cYbb8SIESNi1KhRcd1118UBBxwQffv2jS+++GKtt7HFFlvEeeedF6ecckrU1tbGr371q4ZyubKePXvGvHnzVilwe+65Z1RVVcXee+8dW2yxRcPlV111VYwcOTKOPvro2GyzzWLw4MExaNCgtY7hpJNOik8++SSOP/742GqrrWLo0KHxP//zP1ktk759+8bChQtj+PDhMXv27PjqV78aN910U3z1q1+Nu+++O6699tq45557omPHjnHTTTdF586d1zquSy65JO6888446aSTYsGCBdGxY8c45ZRT4tBDD41PP/00Ro4cGfPnz4+vfOUrccopp0SvXr3WmgUAa1KSrH4cEwAAAGwEjq0BAAAgFQopAAAAqVBIAQAASIVCCgAAQCoUUgAAAFKhkAIAAJCKovke0gULlkR9/bq/gaZ9+1Yxb97ivN1mvvMKkVnseYXIbG55hcgs9rxCZDa3vEJkFnteITKbW14hMos9rxCZxZ5XiMzmlleIzGLPK0Rmc8srRGax521oZmlpSWy99VZrnFY0hbS+PllvIV1xvXzfbr4V+xjd5+LLK0RmsecVIrO55RUis9jzCpHZ3PIKkVnseYXILPa8QmQ2t7xCZBZ7XiEym1teITKLPS/XTIfsAgAAkAqFFAAAgFQopAAAAKRCIQUAACAVCikAAACpUEgBAABIhUIKAABAKhRSAAAAUqGQAgAAkAqFFAAAgFQopAAAAKRCIQUAACAVCikAAACpUEgBAABIhUIKAABAKhRSAAAAUqGQAgAAkIrytAcAAACwKWjXZssoa1m2zutUVLRe67S66rqYv+jLfA9rk6aQAgAAbICylmUx65dvZj1/x+8fkMfRNA0O2QUAACAVCikAAACpUEgBAABIhUIKAABAKhRSAAAAUqGQAgAAkAqFFAAAgFQopAAAAKRCIQUAACAVCikAAACpUEgBAABIhUIKAABAKhRSAAAAUqGQAgAAkAqFFAAAgFQopAAAAKRCIQUAACAVCikAAACpUEgBAABIhUIKAABAKhRSAAAAUqGQAgAAkAqFFAAAgFQopAAAAKRCIQUAACAVCikAAACpUEgBAABIhUIKAABAKhRSAAAAUqGQAgAAkAqFFAAAgFTkvZDecccdsccee8Q///nPfEcDAADQhOS1kP7tb3+Ld955J7bffvt8xgIAANAE5a2QVldXx3XXXRcjRoyIkpKSfMUCAADQROWtkN52221x3HHHxQ477JCvSAAAAJqwkiRJklxD3n777bj11lvj17/+dZSUlMQ3vvGNGD16dOy+++75GCMAAEBRmPXLN7Oet+P3D8jjSJqG8nyEvPXWW/HRRx/FUUcdFRERM2fOjKFDh8YNN9wQhxxyyAZlzJu3OOrr192NKypax5w5lTmPt1B5hcgs9rxCZDa3vEJkFnteITKbW14hMos9rxCZzS2vEJnFnleIzGLPK0Rmc8srRGax5xUisynmVVS0zvl2M7nNYrjP+cgsLS2J9u1brXFaXgrpOeecE+ecc07D3/aQAgAArFu7NltEWct1V7J1leC66tqYv2hpvoe1UeWlkAIAABSTdm22jLKWZeu8zvr2eNZV18X8RV/mc1irKGtZHrP+30tZz9/xosPzOJp0FKSQ/ulPfypELAAAwAYpa1kWM3/x15wytv3hPnkaDWuT1+8hBQAAgA2lkAIAAJAKhRQAAIBUKKQAAACkQiEFAAAgFQopAAAAqVBIAQAASIVCCgAAQCoUUgAAAFKhkAIAAJAKhRQAAIBUKKQAAACkQiEFAAAgFQopAAAAqVBIAQAASIVCCgAAQCoUUgAAAFKhkAIAAJAKhRQAAIBUKKQAAACkQiEFAAAgFQopAAAAqVBIAQAASIVCCgAAQCoUUgAAAFKhkAIAAJAKhRQAAIBUKKQAAACkQiEFAAAgFQopAAAAqVBIAQAASIVCCgAAQCoUUgAAAFKhkAIAAJAKhRQAAIBUKKQAAACkQiEFAAAgFQopAAAAqVBIAQAASIVCCgAAQCoUUgAAAFKhkAIAAJAKhRQAAIBUKKQAAACkQiEFAAAgFQopAAAAqVBIAQAASIVCCgAAQCoUUgAAAFKhkAIAAJAKhRQAAIBUKKQAAACkQiEFAAAgFeX5CFmwYEFceuml8dlnn0XLli1jp512iuuuuy7atWuXj3gAAACaoLzsIS0pKYmzzz47nn322XjyySdjhx12iJ///Of5iAYAAKCJykshbdu2bfTq1avh727dusX06dPzEQ0AAEATlfdzSOvr6+Ohhx6Kb3zjG/mOBgAAoAkpSZIkyWfgtddeG7NmzYo77rgjSkt9ZhIAAJCOmb/4a07zb/vDfRpdNuuXb2ad1/H7BzTO+38vZZ930eFZz1ss8vKhRiuMGjUqPv300xg9enTGZXTevMVRX7/ublxR0TrmzKnMZYgFzStEZrHnFSKzueUVIrPY8wqR2dzyCpFZ7HmFyGxueYXILPa8QmQWe14hMptbXiEyiz2vEJlp51VUtM7L7a58m/nILGTe+qT1vCktLYn27VutcVreCumtt94aU6dOjXvvvTdatmyZr1gAAACaqLwU0g8++CBGjx4dO++8cwwZMiQiIjp16hR33nlnPuIBAABogvJSSHfbbbf4xz/+kY8oAAAAmgmfOgQAAEAqFFIAAABSoZACAACQCoUUAACAVCikAAAApEIhBQAAIBUKKQAAAKlQSAEAAEiFQgoAAEAqFFIAAABSoZACAACQCoUUAACAVCikAAAApEIhBQAAIBUKKQAAAKlQSAEAAEiFQgoAAEAqFFIAAABSoZACAACQCoUUAACAVCikAAAApKI87QEAAACQu3ZttoiyluuueBUVrdc6ra66NuYvWprvYa2TQgoAANAElLUsj9l3PJP1/B0u7JPH0WwYh+wCAACQCoUUAACAVCikAAAApEIhBQAAIBU+1AgAgGanbdutokWLde+bWdenkdbU1MfChUvyPSxodhRSAACanRYtSmPi7+dmPX/fk7fJ42ig+XLILgAAAKlQSAEAAEiFQgoAAEAqFFIAAABSoZACAACQCoUUAACAVCikAAAApEIhBQAAIBUKKQAAAKlQSAEAAEiFQgoAAEAqFFIAAABSoZACAACQCoUUAACAVCikAAAApEIhBQAAIBUKKQAAAKlQSAEAAEiFQgoAAEAqFFIAAABSoZACAACQCoUUAACAVCikAAAApEIhBQAAIBV5K6Qff/xxnHzyydG7d+84+eST45NPPslXNAAAAE1Q3grpiBEj4tRTT41nn302Tj311Lj66qvzFQ0AAEATlJdCOm/evHjvvfeif//+ERHRv3//eO+992L+/Pn5iAcAAKAJykshnTFjRnTs2DHKysoiIqKsrCw6dOgQM2bMyEc8AAAATVBJkiRJriFTp06Nyy67LJ566qmGy/r16xc333xz7L333huck9TWRUl5WdbjWH3+pLY2SsrLs85bU0aumY3zaqKkvEUOeavOX19bHaXlLbPOW33+XPMKkbn6/HW11VGWQ97q8+eaV4jMQufV1lVHeVn2eavPn2teITJXn7+mrjpa5JC3+vzVddXRMsf7vHpGdV1NtCzL/vVh9fnzn1cbLcuyfz1c0/z5ziz+vLpoWZb9dm9NGblmbmp5hRljfbQsy/5/9qvPn/+8JFqWlWSdt6aM2rokynPIXH3+fOfV1SVRluN9Xj0j18zV56+vS6I0h7zV56+vTaK0PIe81ebPNW9NGUltEiU5ZK4+f1JbHyXl2a8rq8+fa14hMgufl+8+lVvehmTk1tb+f9ttt13MmjUr6urqoqysLOrq6mL27Nmx3XbbbXDGvHmLo337VjHn7geyHkfF+afFnDmV//d3ReuYM/qerPMiIirOO7dR5qy7b8w6r+P5lzfKm37n97PO2/57v2yUN+3247PO6zzsiUZ57959XNZ5ERH7nj++UeZr9/bPOu/gcyY0ynvhV8dknXfU2U81ypswpm/WeRER/c+a2Cjz4bF9ss4b/J1nGuX9+r5vZZ13xpnPNcq75/7eWeede/qzjfJu/W32eRERPzi1cebI32efedXJjfN++Gj2j8kvBjV+TE58Ivu8iIhHjm+c2feJ87LOm3j86EZ5/cb9JOu8pwf8tHHe49m/Hj49cNXXwxWZxzx2W9aZT51wcaMxHvNo9tuBpwY13gb0f3Rs1nkTBn2ncd4jD2adFxEx4cRvryHz4RzyBjfKO+6RJ7LOG3/i8Y3yjn/kmazznjixzxqfNwMf/XPWmY8POqLRGAc9+kbWeY8O6tUob/CjU7POe3jQ1xrlnfHYp1nnRUT8+oSdGr/GPj4967yrBm7feLvy2Oys8849oUOjvD88OjfrvIiIkwZt0+i5sy4VFa0zvv7L98/JZmgREXHY6RWN7vPk/85+Ge4/tPEyfP+uWVnnRUR0uaBjo8x/3TIz67xOw7dtlDfz5x9lnbftj3Yt6GOcRmbaeRUVrWP2nY/ndJsdvjewoeutSV4O2W3fvn3sueeeMWHChIiImDBhQuy5557Rrl27fMQDAADQBOVlD2lExDXXXBOXX3553HXXXfGVr3wlRo0ala9oAAAAmqC8FdLOnTvHww9nf4gQAAAAzUvevocUAAAAMqGQAgAAkAqFFAAAgFQopAAAAKQibx9qBABkpqqmJiac+O2cMwBgU6WQAkBKKhdWRWVUrXV6Ib60HQCKiUN2AQAASIVCCgAAQCoUUgAAAFKhkAIAAJAKhRQAAIBUKKQAAACkQiEFAAAgFQopAAAAqVBIAQAASIVCCgAAQCoUUgAAAFKhkAIAAJAKhRQAAIBUKKQAAACkojztAQAAxauqpjbGn3h8TvMDwNoopADAWlUuXBqV65heUdE65sxZ1zUAYO0csgsAAEAqFFIAAABSoZACAACQCoUUAACAVCikAAAApMKn7AIAQI5qa+rjsNMrcpofmiOFFAAAcrRg4ZJ1TvcVSbBmDtkFAAAgFQopAAAAqVBIAQAASIVCCgAAQCoUUgAAAFKhkAIAAJAKhRQAAIBUKKQAAACkQiEFAAAgFQopAAAAqVBIAQAASIVCCgAAQCoUUgAAAFKhkAIAAJAKhRQAAIBUKKQAAACkQiEFAAAgFQopAAAAqShPewArq6uuiYrzT8tpfmgOamqq4owzn8tpfgAAWJe66tro8L2BOWesS1EV0vmLqiJi7W+UKypax5w5lRtvQFCkFi6siYi1/wPGugIAQK7mL1q63utsyPvO0tKStU/LeFQAAACQBwopAAAAqVBIAQAASIVCCgAAQCoUUgAAAFKR86fsXnvttfH6669Hy5YtY8stt4wf//jHsc8+++RjbAAAADRhOe8hPeyww+LJJ5+M8ePHx7nnnhs/+MEP8jEuAAAAmric95AeeeSRDb9369YtZs6cGfX19VFa6mhgAAAA1i6vrfHBBx+MI444QhkFAABgvda7h3TgwIExffr0NU577bXXoqysLCIinnrqqXjyySfjwQcfzGog7du32qDrVVS0zio/F/m+zeaWV4jMYs8rRGameRt7XWmKy3Bj5xUis7nlFSIz7bxNYbvn9WbTyytEZrHnZZO5KdynjX17xb5M0n6MC5FZ7Hm5Zq63kD7++OPrDfnjH/8Yt956a9x3332xzTbbZDWQefMWR319ss7rVFS0jjlzKjc4M18Le+XbzEdmc8srRGax5xUiM9Pn/sZeVza1ZViMeYXI3JTyCpFZ7Hnrk+m6vGKeXG1KrzeFyGwOeYXILPa81TPXJ5v1L595m8IyLMYxpvkYFyKz2PM2NLO0tGStOyBzPof0xRdfjBtuuCHGjh0bnTp1yjUOAACAZiLnQnrFFVdEixYt4qKLLmq47L777outt94612gAAACasJwL6f/8z//kYxwAAAA0Mz4OFwAAgFQopAAAAKRCIQUAACAVCikAAACpUEgBAABIhUIKAABAKhRSAAAAUqGQAgAAkAqFFAAAgFQopAAAAKRCIQUAACAVCikAAACpUEgBAABIhUIKAABAKhRSAAAAUqGQAgAAkAqFFAAAgFQopAAAAKRCIQUAACAVCikAAACpUEgBAABIhUIKAABAKhRSAAAAUqGQAgAAkAqFFAAAgFQopAAAAKRCIQUAACAVCikAAACpUEgBAABIhUIKAABAKhRSAAAAUqGQAgAAkAqFFAAAgFQopAAAAKRCIQUAACAVCikAAACpUEgBAABIRXnaAwDYENU1VXHVyc/mND8AAMVFIQU2CYsW1kREzVqnV1S0jjlzKjfegAAAyJlDdgEAAEiFQgoAAEAqFFIAAABSoZACAACQCh9qBAAAZKyuuj46Dd82p/lBIQUAADI2f9GSdU73CfhsCIfsAgAAkAqFFAAAgFQ4ZBcAAEhdXXVdbPujXXOan02PQgoAAKRu/qIv1zndOalNk0N2AQAASIVCCgAAQCoUUgAAAFKhkAIAAJCKvBXSN954I/bcc8944IEH8hUJAABAE5aXT9ldvHhx/PznP4/DDjssH3EAANCgpqY+Thq0Tc4ZQPHJSyG98cYbY+jQofHnP/85H3EAANBg4cIl65zu60Bg05VzIX3ppZfiiy++iD59+iikAJCyqpramHDi4JzmB4CNZb2FdODAgTF9+vQ1TnvmmWfilltuibFjx+Y8kPbtW23Q9SoqWud8W5nK9202t7xCZBZ7XiEyM83b2OtKMSzD5nifM8mrrquJpwf8NOus6rqa1J/XaWSmnbexn9ebtyiPzYt8jJ43xZmZdl5zXEZp3F7aj0ux5xUis9jzcs1cbyF9/PHH1zpt0qRJMWfOnBg8ePl/YhcsWBAvvvhiLFy4MC688MKMBjJv3uKor0/WeZ1MD8fI18Je+Tbzkdnc8gqRWex5hcjM9Lm/sdeVQi/D9WmK97kwz5uqtV53w5bh/81fiPEV4+OyKa3LaWSmve4VIrM55BUi07qS37xifExWz1yftJfhxs4rRGax521oZmlpyVp3QOZ0yG6PHj3i9ddfb/j78ssvj6997Wtx2mmn5RILAABAM+B7SAEAAEhFXj5ld4Ubb7wxn3EAAAA0YfaQAgAAkAqFFAAAgFTk9ZBdAPKnqqY6Jh4/Oqf5AQCKmUIKUKQqFy6Lyli21umF+Oh2AICNySG7AAAApEIhBQAAIBUKKQAAAKlQSAEAAEiFQgoAAEAqFFIAAABSoZACAACQCoUUAACAVCikAAAApKI87QEATU91TVX84NRnc84AAKBpU0iBvFu0sCYiatZ5nYqK1jFnTuXGGRAAAEXJIbsAAACkwh5SAIBNSFVNXfz6hJ1yzgAoBgopAMAmpHLhl7G+Ex6cFgFsKhyyCwAAQCoUUgAAAFLhkF0gqmuq4tzTs/+aFl/RAgBANhRSYL1f0+JcJAAACkEhBQBo5qpr6uKqgdvnND/5VVtdH/sP7ZDT/LApUEgBAJq5RQu/XOd0R8psfAsWLVnndI8JTYVCCgBQQFU1dfHwoK/lND9AU6WQAgAU0Pq+N9SeLqA587UvAAAApMIeUgCyUlVTE08PvDyn+QGA5k0hBSArlQurojLW/h20DkMEANbHIbsAAACkQiEFAAAgFQopAAAAqXAOKQBFo6qmJp464eKc5gcANh0KKQBFwwclAUDzopACAEATV1tdH10u6JhzBuSbQgoAAE3cgkVL1nsdR6GQBh9qBAAAQCrsIQUANpqqmtp44sQ+Oc0PQNOhkAIAG03lwqWxrgMCHTII0Lw4ZBcAAIBUKKQAAACkQiEFAAAgFQopAAAAqVBIAQAASIVCCgAAQCoUUgAAAFKhkAIAAJAKhRQAAIBUKKQAAACkojztAQDApqKqpiYmDPpOTvMDAP9HIQWADVS5sCoqo2qt0ysqWsecOZUbcUQAsGlzyC4AAACpUEgBAABIhUIKAABAKvJyDun9998fDz74YLRo0SLKyspi3Lhx+YgtSnXVNdHx/Mtzmh8AAIA8FNLnnnsunnnmmXjkkUeiVatWMWfOnHyMq2jNX1QV4QMtADYJVTU18dSgc3OaHwAonJwL6ZgxY+Liiy+OVq1aRURERUVFzoNqTuqqq2P77/0yp/kBWDOfigsAxS3nQjpt2rR4991347bbbovq6uoYMmRInHTSSfkYW7Mwf9GyiFi21uneLAEAAE3VegvpwIEDY/r06Wuc9tprr0VdXV3MmDEjfvvb38aCBQvilFNOiV122SV69j62G78AACAASURBVOyZ0UDat2+1QderqGidUW4+ZHqb+R7jxr7Phbi9Yl8mTfE+p337aWQ2x3Wl2JdhMSwjeYXPLPa8NG4z7XW5EJnF/hrbHJeh+1x8eYXILPa8XDPXW0gff/zxdU7ffvvto3///lFaWhrt27ePgw8+OKZMmZJxIZ03b3HU1yfrvE6mewvztbAzvc187tFM4z6vfHuFWIbFOMZN7T6vT9rPwzQym+K6sj5pL8ONnVeIzOaWV4jMYsgrxvU5zXW5EJlpv8ZuyO01tWW4sfMKkdnc8gqRWex5G5pZWlqy1h2QOR+y279//3jllVeiZ8+e8eWXX8bkyZPjm9/8Zq6xAAW1rGZZ/GLQMznNDwBAbnIupGeeeWZcddVVccwxx0RExPHHHx9f//rXcx4YQCF9sbA6Itb+oWDO3wYAKLycC+nmm28eN998cz7GAgAAQDOScyEFAGhKqmrq4tFBvXKav7mrrqmPc0/okNP8QPOgkAIArKRy4ZexrgP2HdK/fosWLlnndMsQWKFJF9K66pqoOO/cnDMAAADIvyZdSOcvqoqIqrVO9985AACA9JSmPQAAAACaJ4UUAACAVCikAAAApKJJn0MKADR9VTW18figI3KaH4B0KKQAwCatcuFSX9MCsIlyyC4AAACpUEgBAABIhUIKAABAKhRSAAAAUuFDjdjk1VQvi6POfiqn+QEAgI1PIWWTt3BRdURUr3W6T1cEAIDi5JBdAAAAUqGQAgAAkAqFFAAAgFQ4hxQ2gprqZTH4O8/kND8AADQ1CilsBD54CQAAGlNI2ehqq5fFwedMyGl+AABg06eQstEtsLcQAAAIH2oEAABAShRSAAAAUuGQ3SamtnpZdB72RE7zAwAAbAwKaRPj/EwAAGBT4ZBdAAAAUmEPKaympnpZ9D9rYs4ZNC9VNcvikeOfyTkDAKA5UUhhNQsd9kwWKhdWR6XnDQBARhyyCwAAQCoUUgAAAFKhkAIAAJAKhRQAAIBUKKQAAACkQiEFAAAgFQopAAAAqVBIAQAASIVCCgAAQCoUUgAAAFKhkAIAAJAKhRQAAIBUlKc9AIpbbfWy2Pf88TlnAAAArE4hZZ0WLKqOiOp1XqeionXMmVO5cQYEAAA0GQ7ZBQAAIBUKKQAAAKlQSAEAAEiFQgoAAEAqFFIAAABSoZACAACQCoUUAACAVCikAAAApEIhBQAAIBUKKQAAAKkozzXg448/jquvvjq++OKLqK6ujn79+sWwYcPyMTYAAACasJwL6c033xy9e/eO0047LZYsWRL9+/ePww8/PLp27ZqP8QEAANBE5XzIbklJSVRWVkZERFVVVZSUlES7du1yHhgAAABNW86F9Morr4ynn346Dj300PjGN74RQ4cOjU6dOuVjbAAAADRhJUmSJOu6wsCBA2P69OlrnPbaa6/FbbfdFl/5ylfi7LPPjtmzZ8fpp58eN910U+y7774FGTAAAABNw3oL6fp07949nn/++Wjfvn1ERIwYMSJ22GGHOPvsszPKmTdvcdTXr3soFRWtY86cyqzHWui8QmQWe14hMptbXiEyiz2vEJnNLa8QmcWeV4jM5pZXiMxizytEZrHnFSKzueUVIrPY8wqR2dzyCpFZ7HkbmllaWhLt27da87RcB9CpU6d45ZVXIiJi8eLFMXny5Nhtt91yjQUAAKCJy/lTdm+44Yb46U9/GmPGjIna2tro169fHH744RnnlJaW5PV6+b7dNDOLPa8Qmc0trxCZxZ5XiMzmlleIzGLPK0Rmc8srRGax5xUis9jzCpHZ3PIKkVnseYXIbG55hcgs9rwNyVzX9JwP2QUAAIBs5HzILgAAAGRDIQUAACAVCikAAACpUEgBAABIhUIKAABAKhRSAAAAUqGQAgAAkAqFFAAAgFQopAAAAKRCIQVYzbRp09IeAgBQAF988UUsWbIk7WGwkqIupJMmTYqRI0fGBRdcEBdccEGMHDkyJk2alNfbuOqqq7Kab8GCBfHjH/84zjrrrHjwwQdXmTZs2LCsMv/5z3/GBx98EBERn3zySdx3333x2muvZZW1JldffXVO868YW0RETU1N3HbbbXHGGWfEjTfeGEuXLs11eLFkyZL429/+FosXL85q/hdffDFqampyHsfK5s+f33DfZs6cGc8++2xOZaWmpibuvffeOO6446JHjx5x4IEHxre//e3405/+lNM4p0+fHmPGjImf/exn8bOf/SzGjBkT//rXv3LKXN3ZZ5+d1XzFvq4sXbq00c93v/vdqKqqyup5/eqrrzb8XllZGZdcckkcffTRMWzYsJg7d25WY1zd559/Hs8991x8/PHHWc2f73Wlvr4+nnnmmXj77bcjIuK5556LkSNHxu9+97uor6/POnfWrFlx3333xc9+9rMYNWpUPPLII7Fs2bKs8zbGehKx6nMgE5MnT44nn3wy5syZs8rljz/+eFZ506dPj2HDhsXFF18cc+bMiWuvvTb222+/OOWUU/J2v88999y85EREvP7663HnnXfm9Hr4yiuvxKJFiyJi+ZvOK6+8Mo499ti45JJLYv78+RnnPfDAA1nNty5//etfY/r06RERMWXKlBgzZky88sorOWVOmTIlfvrTn8YFF1wQF110Udx2222NnkeZ2BjvvyKyew9WXV0dd999d1x11VXx5z//eZVpI0eOzHoshX4Pls91ZdGiRTFp0qSYN29e1hmPPvpow3N75syZccYZZ8R+++0Xp556anz22WcZ5+V7uzJt2rQ455xz4uqrr45FixbFeeedF927d4+TTz45q/dhX3zxRYwYMSL222+/6NWrV/To0SOOOOKIuP/++3Me69SpU+P555+P559/PqZOnZpz3prk+x/lK14ns1VZWdmQsWjRovjjH/8YH330UdZ5JUmSJDmNqEDuuuuueOaZZ2LAgAGx3XbbRUTEjBkzYty4cdG7d+/43ve+l5fbOeKIIxq9oG2Iiy66KDp16hTdunWLhx56KLbaaqv45S9/GeXl5TFgwIAYN25cRnn3339/jB07Nmpra2Po0KHxxBNPxD777BNvvPFGnH766fHtb387o7ybbrqp0WUPP/xwDB48OCIiLr300ozyIiIGDhzY8MboF7/4RXz44YcxePDgePbZZ6NFixYZbwiuvvrq+P73vx/t2rWLyZMnx7Bhw2LrrbeO+fPnx8033xyHHHJIRnl77rlntGnTJo499tgYNGhQdOnSJaP5V/e73/0uRo0aFa1bt44RI0bEddddF7vvvnu8//77cdFFFzUsy0xcccUVUV5eHocddlhMnDgxdt5559htt93iv/7rv2LQoEEZP84Ryx/XO+64I44++uhV1pUXXnghvve972U0znUVsD59+sRLL72U8fiKfV3p0qVLlJSUxJpeCktKSuLvf/97RnkrryfXXXdd1NfXx6mnnhpPPfVUfPrpp/HLX/4yo7yIiPPOOy+uv/76aNeuXbzwwgsxYsSI2GuvveIf//hH/PCHP4zjjz8+o7x8ryvXXXddTJ06NWpra+OQQw6JN998M4466qh4/fXXY9ddd42f/OQnGWeOHz8+br311ujSpUu8/fbbcdBBB8WSJUvigw8+iNGjR8cee+yRUV4+15P1yWa7Mnbs2HjooYdi1113jSlTpsSIESOid+/eEbHqcyoT3/3ud+PQQw+NxYsXx8SJE6N///4xaNCgmDhxYrz++utx1113ZZR38cUXN7rs5ZdfjsMOOywiIm677baM8k4++eT4/e9/HxER48aNizvvvDOOPvro+Mtf/hIDBw6Ms846K6O8iIj+/fvH+PHjo7S0NH7yk5/ElltuGf37949XXnkl3nvvvbjzzjszyuvatWuUlZXFQQcdFIMGDYojjjgiysrKMh7XCnfccUc89thjUVJSEkOHDo3f/OY3ceCBB8bkyZOjX79+cf7552ecOWbMmHjiiSeiZ8+e8Ze//CV69eoVpaWl8fzzz8ctt9wSBxxwQEZ5G+v9V0R268qVV14ZS5cuja5du8ajjz4aBx10UPz4xz+OiOzXlXxvV/K9rowcObKhvL/zzjtxwQUXxLbbbhvTp0+Pm2++OQ499NCM8iKWrysTJkyIiIjvf//70a1btzjuuOPi5ZdfjnHjxsV9992XUV6+tyvf/va3o0+fPlFZWRkTJ06MAQMGxIABA+LPf/5zjBs3LuMief7550fXrl3j8MMPjyeffDK23nrrOPDAA+P222+PffbZJy666KKMxzhlypT40Y9+FJttttkq68qyZcvi5ptvjn333TfjzLXJZl2ZOnVq/OAHP4hZs2bFEUccEddcc020a9cuIrJfVyIinn766bjqqquitLQ0rrvuurj77rujQ4cO8f7778dPfvKT6NOnT+ahSZH65je/mVRVVTW6fOnSpcnRRx+dUdaBBx64xp9evXole+21V1bjO+644xp+r6+vT6655prkrLPOSqqqqpLjjz8+47xjjz02WbJkSTJnzpyka9euyYwZM5IkSZJ58+Zllbfvvvsmw4cPT26//faGnx49ejT8no2VxzFgwIBk8eLFSZIkSU1NTXLMMcdknHfsscc2/H766acn7777bpIkSfLRRx8lAwcOzGp8f//735ORI0cmvXr1SgYOHJg88MADyaJFizLOSpIk6d+/fzJ79uzk448/Trp375589NFHSZIkyezZs5P+/ftnldm3b9+G32tqapIhQ4YkSZIkCxcuTHr37p1V5re+9a1k3rx5jS6fN29e8s1vfjOjrD322CPp0qVLssceezT8rPi7S5cuWY2v2NeVyy+/PLnyyiuTysrKhsuOPPLIjHNWWHkMxx13XFJdXd3wd7bPm5WX4cknn5x8+umnSZIkydy5c1dZjzIZYz7Xlb59+ya1tbXJ4sWLk+7duzcsy2XLlmX12pAkSdKvX7+G5/Vnn32WnHfeeUmSJMlrr72WnHrqqRnn5XM9SZIkGTVq1Bp/brzxxmS//fbLOK9///4Ny+3DDz9MevfunTzyyCNJkiRZPa+TZNXnzde//vVVpmXzvDn44IOTK664InnssceSxx57LHn00UeTXr16NfydqZXv10knnZR8/vnnSZIkyYIFC7JeV/r169fw+4ABA1aZtvLyyGSM8+bNS8aMGZP0798/+frXv56MGjUq+fDDD7Me35dffpnMnTs36datWzJ79uwkSZKksrIy63Wlb9++yZdffpkkyfLn83e+850kSZLk73//e1bb0ny+/0qS/L8HW/m5sXTp0uT8889PrrjiiqS+vj7rdSXf25V8rysrP5fPOuus5NVXX02SJEnee++95MQTT8w4L0mSVd5zrL6uZHOf871dWXl9Peyww3Ie3+qvKSeddFKSJElSVVWVfOtb38pihMvHMWnSpEaXv/XWW1m93jzwwANr/TnggAMyzjvllFOSF198MVmwYEFy++23J3369EmmT5/eMPZsDRgwIJk1a1by0UcfJd26dUv++c9/JkmyfFt9wgknZJVZtIfsJkkSJSUljS4vLc18yEmSxNixY+ORRx5p9LPiPwWZqq6ubvi9pKQkRowYEbvvvnucc845WR1SVlpaGltuuWVss802scMOO8S2224bERHt2rVb43JYn8ceeyxmzZoVW221VXzve9+LCy+8MFq3bh0XXnhhXHjhhRnnRSxfjisOYywrK4utttoqIiLKy8ujvLw847yVl9OSJUuia9euERGxyy67ZHXYR0lJSXTp0iV+8pOfxCuvvBJnn312vPjii3H44YfH8OHDM84rLy+PioqK2HnnnaNNmzaxyy67RERERUVF1v8hLysra7jfixcvbvi9TZs20aJFi6wy6+vr1/g83nrrrde4129dKioq4tVXX43333+/4efvf/97vP/++9GhQ4esxlfs68oNN9wQRx99dJx55pkNe4CzyVmhuro6pk2bFh9++GGUlJSs8rhm8/oVEVFVVdVw6Gt9fX3suOOOERHRvn37jB/jiMKsK2VlZbHlllvGFltsEa1atYqIiJYtW2Z9n8vKyhqe1zvssEPMmDEjIiIOOuigWLBgQcZ5+VxPIpbvUdlss81iyy23XOVnq622yvr5s2K5de7cOX7zm9/E2LFj43e/+13WeSvPt9dee6112oZ68sknG06t6NOnT5xwwgmx5ZZbxsCBA2PgwIE5ja+mpiY6deoUERFt27bNapsSsfzxnDJlSkQsfz1bcUjjkiVLsjp8vKSkJNq1axff+c534sknn4y77rorlixZEkOGDIkhQ4ZknNeiRYvYYoston379tG6deuoqKiIiOWPfbb3uaysLLbYYouIiPjKV77ScJ+7dOmyyuvvhsrn+68Vefl8D1ZXV9fw++abbx633357LF26NC655JKsTxHI93Yl3+vKyubOnRsHH3xwRCzfK5nNYxwRseOOOzYcHr/jjjvGJ598EhGR9aHe+d6u1NXVxbJly2L+/PnxxRdfNBxevHTp0qzeO5SUlMTChQsjIuLf//53w3Nls802y3rdW7p0aey///6NLu/Ro0dUVVVlnHf99dfHX//615g6dWqjn2we5y+//DKOOOKIaNu2bcP7/zPOOCM+//zznN7nRER06NAhdtlll9h2221jt912i4jl2+qV189MZPcIbAQDBgyIwYMHx4ABA2L77bePiOXnw4wbNy7jw9O+9rWvxYIFC9Z4+EDHjh2zGt8OO+wQb731VvTs2bPhsssuuyxuvfXWuPfeezPOW/lF9Ic//OEq07IpZ7vuumv8+te/jnvvvTf+8z//M6655pqcn3z/+Mc/onv37g0bq1mzZkXHjh1j2bJlWW0EDjrooLjxxhvj4osvjl69esXTTz8d/fr1i1dffTXatm2bcd7KbypbtGgR/fr1i379+sXMmTMzPix0RcZLL70UlZWVUVJSEs8++2z07t07Jk2alPWGuX///nHSSSdFjx494tVXX204DCiXcwsPOeSQOPvss+Okk05aZV35wx/+EF//+tczyurVq1d88MEH0atXr0bTVvzDIFPFvq5ERBx55JHRrVu3GDlyZEycODHrF9SI5eXxnHPOaXg+rlhPFi9enPXzpm/fvjF8+PAYPnx4HH300Q3nIb/88svx1a9+NeO8fK8r2223Xdx0002xZMmS6Ny5c1x//fVx7LHHxiuvvBLbbLNNxnkREZ06dYq77747Dj300HjqqacaNnh1dXVZPT75XE8iInbffffo3bv3GrcrDz/8cMZ5LVq0iDlz5jQUlA4dOsSvf/3rOOuss+LTTz/NOC9i+Zv1xYsXR6tWrVZZ1xYuXJjVP9XatWsXt912W4wfPz5OO+20uOSSS3Larnz88cdx4oknRpIk8dlnnzWMNSL7dfnHP/5xDBs2LHr06BEVFRVx0kknxYEHHhhTpkzJ6jz41f9Z0bVr1+jatWtcccUV8cc//jHjvNatW8eDDz4YixcvjrZt28aYMWNiwIAB8fLLL8fmm2+ecV7E8n82XHXVVXHIIYfEs88+2/AGedmyZVktx3y+/4rI/3uwbbbZJt5///2GvLKysrjlllvisssuW+WzLjKR7+1KvteVWbNmxU033RRJksSiRYuirq6uYR3OtoSPGDEiLrzwwhg7dmy0adMmBg8eHHvvvXfMmDEjq3N7871dOfbYY6Nv375RW1sbw4YNi2HDhsUee+wRkydPjqOOOirjvDPOOCOOO+642GuvvWLq1Klx5ZVXRsTy918rnueZ6tSpU4wePTqGDBnS8L514cKF8dBDD2WV2blz5/jud78bnTt3bjQtm/OZly1btspz5ZhjjomWLVvGmWeeGbW1tRnnrbDyNviMM85Y67RMFO05pBHLT6p/+umnG07+33777aNPnz4Znw9RXV0dZWVlOZ33sbqFCxdGSUlJtGnTptG0Dz/8MP7jP/4jo7zHHnssevfu3bDXcYWPPvoofv/738cVV1yR9Vjff//9uPrqq+Pzzz+P119/Peuctfniiy/io48+im7dumU0X3V1ddx0003xxBNPRNu2bePzzz+P8vLy6NWrV1xzzTWxww47ZJR32WWXxahRozKaZ13efffdGDFiRJSWlsb1118fv/rVr+L555+PzTbbLG699daG/1BmasUeyL333jsOPPDAnMdZX18f48ePj4kTJzZaV44//visS1C+bErrSkTEM888E2+88UaMGDEip5zVLV26NObOnZvx8zpi+Yb+vvvui7Fjx8aCBQuipqYmttpqqzjmmGPiBz/4QWy99dYZ5eV7XZk/f36MHj06SkpK4oILLohnnnkmHnjggejUqVNceeWVWd3nuXPnxg033BD/+Mc/4mtf+1pceuml0a5du1i4cGFMmTKl4VysDZXv9eTVV1+NHXfccY33bfLkyWv8r/m6vPDCC1FRUdHoHz8LFiyI//qv/8rqvP+17emaP39+zJ07N3bfffeMM1eYNWtWXH311fHOO+/EG2+8kVXGm2++ucrfe++9d2y11VYxZ86ceO6557I6pz5i+bo2YcKE+PDDD6Ouri6233776Nu3b8M5Xpm45ZZbstq7szaffPJJ3HzzzVFaWhpXXHFF/P73v4/f/OY3scMOO8QNN9wQe++9d8aZX375Zdxzzz0N25Vzzjmn4Z8R//rXv7I6l++tt95a47qS6fuviPy/B/vkk0+iRYsWjf4ZlyRJvPzyy3H44YdnnFnI7Uo+1pU77rhjlb9POeWUaN++fcyaNStuvfXWuPHGG7Me32uvvbbKunLYYYc17HHPRL63KxHL379GLN/bP3369HjmmWeiU6dO8a1vfSurvGnTpsUHH3wQXbp0iZ133jnn8c2bNy9uueWWmDhx4iqX9+nTJ4YPH57xP2THjx8fe+211xrfFz311FNxzDHHZJR33XXXxeGHH95onXjxxRfjyiuvzLoTjB49Ok477bSGfyCuMG3atLj99tuz+qyMoi6k5E9tbW3MmTMnqw1yoX355Zfx2WefNbwYZvrmemNasGBBtGnTJvWSR/O1ePHiqK2tzeooAgAg/1YcDmzbnB3vqpuJ8vLyoiyjERFbbrlldOnSJfbee++iLqMRy89P2pTKaLZfP7Em2X7ty7oce+yxec3L5/2NyP99zsf9bdWq1SobvHx/l1q+l2G+83LJnD59evzv//5vo3NxiiVv/PjxMXr06Ia9Aivcc889WeUV4iuX8v3VNPm+z4XIzPd9rq6ujlmzZjW6PNvDTSPyP8YXX3yx4Zz6SZMmxU9/+tOsDkWPWP71EKNGjWo4rP9Xv/pVHHfccfGjH/2o4U18Jgr1dUb5/tqXfC7DQowv349Lvse4qTxvVmjbtm20bds2408vX59cv7Kx0Hn5yrWHFJqwTD8mvBBf+/Lhhx+uddqZZ54Zf/nLXzLOXJtsPhY93/d5Y97fiOy/umpTzcs2c/z48XH99ddHRUVFLF68OH7xi19E9+7dIyK7j7/Pd95NN90U/197dx5XU/7/Afx1RUjR8LXMpLF+B2Xrx9jJ2kyJFiV7dlrIEqWRnSRblgpjGlvCuIkmw4y+jDV8x6SJGJSsoWWUUqnz+6PHPd+ustxzPsc91fv5eHjMdBrveZ97zz33fM75fN7vP//8EyYmJjhx4gQmTZqE8ePHC44HsG+5xLo1jRT7HBAQgGvXrjGLyXqfz507h9mzZ4PjOHz55ZfYsGEDmjRpIjieFDlu3LgR58+fx5s3b9CtWzf89ddf6N27N86ePYsuXbpofDNj1qxZaNiwIXJycnD//n20aNECtra2OHHiBF68eFFmm7r3Yd3OCGDf9oX1a8g6P4D9+8I6x/Jw3Lx9ow8ANm/ezL+/2m7ZKEULSKniyraoESHk47zrpMxxHLKysjSKZWZmVqonp+pnoQUZrK2tYWRkVGYlUyF3OVnuL8B+n1nvL4D3DoqFVBtk/RqyjidFzJ07dyIyMhINGzZEbGws5syZg+XLl6NXr16Cquyyjvf7778jIiIC1apVg4uLC1xdXZGdnQ13d3dB8QDg/v372LRpEwBg0KBBWLZsGaZNmyb4Dr5SqYRSqYS+vj7u3r0LNzc3ZGdnY9iwYbLZ5zNnzjCNyXqfN2zYgD179qB169aIiIjAhAkTEBQUhNatWwveZ9Y5njp1CkeOHEFubi569eqF06dPw9DQEGPGjIGTk5PGg6m7d+9i48aNKCwsRI8ePRAaGgodHR20b98eQ4cO1Ti/Z8+eYdy4cQCAsLAwTJs2DQAwduxYwU8gDx06hKioKOTk5GDAgAE4ceIEGjVqhPT0dEycOFHjgQXr15B1fgD794V1juXhuFm5ciX69OmjNrsvLy8Pf/31l6B4YWFhGDhwYKn1rXp6erKIJ2XccjUgjYqKgrW1NZNYq1evhre3N5NYKmPGjMHevXuZxbt8+bKgAgKfKh5QXI1u/fr1TGOyxHqfc3JyRH+Q3yb2WNyzZw8mT55cZsEITQdU9evXR2RkZJml+IUUigAAIyMjhIWFlVlNUUhMlvsLsN9n1vsLANOnT8fXX39d5sWlkCm7rF9D1vGkiMlxHP+edO3aFTt27MDUqVPh6+sri3gA+BZB9erVw86dO+Hi4oK8vDzB8cpqueTv7y+45RJQujXNxIkTUVBQIJt9liImy30uLCzkiwzZ2dnByMgILi4uCAwMFLXPLHNUtXHS19fHl19+yS8R0NPTE1SYSNVSQ0dHB59//jkfQ6FQCFoCw7qdEfC/ti96enpM2r6wfg1Z56fKEWD3vkjxGrLMT4rjJjQ0FOvWrYO9vT1faCk2NhZ+fn6C4imVSixevBimpqYYP348FAoFlEql4HaNrONJGbdcDUh37tzJbEAqtNLZ+7Bey+Xn5yd4/ceniAcUl+1nifWgnvU+jx49mvlrKPZYZNl+Qoq2LxYWFnj06FGZA7RBgwZpHI91uw3W+8x6fwGgSZMmWLlyZZkVXYUMclm/hqzjSRUzMRrQFAAAIABJREFUKysLBgYGAICWLVvihx9+wOTJk/HPP/9oPZ6+vj5SUlL4HrP6+vrYsWMHpk2bhtu3bwvKj3XLJdataaTYZ9YxWe/zmzdvkJeXh+rVqwMAunTpgvXr18PDw0PwTQLWORYVFfEzRFatWsVv5zhOUKsIVf/t6tWrq00Tz8nJETQQYN3OCGDf9oX1ayhFuzPW7wvrHMvDcdO1a1eEhoZi9erViI6OxqJFi0TdWGLdslGKFpCSxeXKERsbG1nGUnF0dGQaj3WOUuwz65i2trZM41WG1/DcuXNcSkpKmb+7evWqqNhyVNn2l+M4bufOnVx8fHyZv/v+++81jsf6NZTiPWEd8+DBg9zly5dLbU9JSeFmzpyp9Xh//PEH9/fff5fanpeXx+3du1fjeBzHcRkZGVxmZmaZvyvr//Uhv/32GxcXF1dqe3p6Oufv769xPCn2mXVM1vu8detW7ty5c6W2x8XFccOGDdM4nhQ5/v7771xOTk6p7UlJSYLON8+ePePevHlTavuTJ0+4CxcuaByvqKiozO1paWncrVu3NI7HcRx3+PBhLjs7u9T2u3fvcqtWrdI4HuvXkHV+HMf+fWGdY3k4bko6ffo0N3ToUK5Hjx6iY3Ecx928eZNzdHTkunXrJst4rOOWq6JGSqUS9vb2TGLdvn1bVA+2TyEkJATTp0+XbTyg+M4SyxLXw4cPx8GDB5nFY73P7u7upfqBiVUejkVCCCGEEPJuWVlZuH37tsZ9qN+FdctGqVpAsohbrgakhBBCCCGEEEIqjvLTUJEQQgghhBBCSIVCA1JCCCGEEEIIIVpRqQakQqrNfcj27duRkZHBPK6c7dix46O2Ee25fPkys1hvV8tjYcyYMUzjsdxfgP0+s95foLjSIEusX0PW8aSIGRUVJet4q1evZhoPYH8s/vDDD0zjSbHPrGOy3mcpPiusc5T7sS3F95TczzdSHDes3xfWOZaH4yY5OZlpvPLw3cwirmwHpGvWrPngH02be7u6uqJfv35YsGABIiMjkZqaKjrPZ8+eYfDgwZg/fz7i4uJEx4uOjkZ2djYAIDAwEJMmTRLcYFeKeKqYH7NNCBYn7NzcXGzYsAFz584FUNxc+bfffhMV8+LFi3w7mhcvXghud3P8+HF8/fXX6NGjB7PXrCxCe2CVhXVrH0CaFkkssd5n1vsLQFAj9Pdh/RqyjidFzJ07d8o6XnloT3bs2DGm8aTYZ9YxWe+zFJ8V1jnK/diW4ntK7ucbKY4b1u8L6xzLw3Eze/ZspvHKw3czi7iyHZAePXqUb7D7rj+RkZEaxfz5559x8OBBdOvWDZcuXYKjoyOsrKxE5blw4UKcOnUKZmZm8PX1hb29PQ4fPiy4n1hwcDD09fVx/fp1nDt3Dra2tlixYoXg/FjGO3/+PPz9/fHs2TO1GwOLFi0SnN/bWJywlyxZgsLCQiQmJgIAGjVqJKoy7vbt27Flyxbs3r0bQHE1MR8fH0GxAgMDsX//fmzfvh0bNmxAUFAQbt++jezsbDx48EBwjm9jWatMirpnqsb1rLDOkXU81vsLyH+fpThu5J6j3OMBle+zJ0VMuceTImZliydFTLnHkyJmZYsnRUy5x2MVtyqjPJjr2bMn3N3d3/vfaHoBz3Ecnjx5gsePH+PRo0cwNDRkUpq5Zs2acHJyQr169bBq1Sps374dgYGB8Pb21njAW7Vq8Vty/vx5ODo6YsiQIaKm3rCMV61aNdSqVQsKhQJ6enr89gYNGmDq1KmCcyyJxQfl9u3b8Pf3x7lz5wAAtWrVUmvYrKmoqCgcPnwYjo6OAIoHuKqnzprS1dVFy5YtAQD79u3DypUrERkZCW9vb4SHh2Pbtm2C8yzp22+/ZRIHAH788UdmsVRYtvYB2O4vwH6fWe8vADRu3JhpPNavIet4UsQcN26crOOtWbOGaTyA/bHo6enJNJ4U+8w6Jut9luKzwjpHuR/bUnxPyf18I8Vxw/p9YZ1jeThu/u///o9pvPLw3cwkrqgupuVMp06duBEjRnARERHc06dPmcR8/vw5t3nzZq5fv37czJkz+UbpKSkpnLm5ucbx7OzsuMjISG7w4MF8U/jBgwcLzo91vDdv3nB79uwR/Pc/5PDhw6JjODo6chzHcTY2NhzHcdzr1685a2trZvE4juOGDBkiKNbkyZO5+Ph4wbkQQgghhBBSkch2yu7bUlJS8OOPP4paCzht2jTo6enh+++/x5YtW3D8+HGkp6eLysvW1hb5+fkICwtDYGAgvv76awCAsbEx7O3tNY7n6+uLX375BY6OjjA2NkZycjK6du0qOD/W8XR0dHD8+HHBf/9DhLxmb+vcuTNCQkKQn5+P2NhYeHh4oH///oLjNWrUCFevXoVCoUBRURGCgoLw73//W1Asf39/NGjQQHAuhBBCCCGEVCQKjpNoMrFI48ePh7e3N1q3bo2nT59iyJAh6NixIx4+fAgbGxtMnz5dcOyCggJERUVh8+bNePr0KW7cuCE4Vl5eHqpXry7475dUWFiIrVu3YubMmUziSWXLli1o2bKlZI/9xSooKMD333+PmJgYcByH/v37Y+rUqfz0ZU09f/4cXl5euHz5MhQKBTp37oy1a9eiXr16jDMnhBBCCCGkcpHtgNTKyoqvQrp9+3b89ddf2LRpE16+fInRo0cLqiB34sQJXLx4ERcvXkRhYSG6deuG7t27Y/DgwYLzzMvLw9GjR/HgwQO8efOG3z5//nxB8caOHYs9e/YIzudtycnJWLBgAVJTUxETE4OEhATExMRgxowZgmN269YNmZmZqFGjBmrWrAmO46BQKHDx4kVmectRbm4uioqKUKtWLW2nIrkdO3ZgypQpH9wmVFRUFKytrZnEkkJaWhoePHiAjh07Co6xfft2ODo64rPPPmOYGRHr7t27aNGixQe3yYEUecn9s0cqnjFjxmDv3r0ICAjAvHnztJ1OuVVUVISzZ8/C3Nxc26l8ElIeNy9fvgQA1K5dm2lcIpxsixqVfOr4xx9/YODAgQCKDx4dHR1BMU+ePIlu3bph8uTJzAqCeHh4oKCgAO3bt4eurq7oeH379sXOnTtha2urVjioZs2aguItWbIELi4uWLduHQCgTZs2mD9/vqgB6eHDhwX/3U+hrEXvBgYG6NixI7p3765xvHHjxsHBwQEWFhayHIzu27fvg/+Nnp4e7OzsPjpmdHR0qcFnWduE2rlzp+iL4tzcXISEhODhw4dYt24d7t69i6SkJP5coalRo0Zh27Zt4DgOtra2qF27Nvr06QMvLy9B8VQtoXr16oXRo0ejQ4cOguKUdOfOnVLbDAwM0LBhQ9GxVRISEmBqairbeGJjenp6IiIi4oPbPlZoaCgcHBxgYGCAefPmIT4+HgsXLkSvXr00ipObm1tq25QpUxAdHQ2O4wR/B7yNxWdv9erVcHNzQ82aNTFu3DjcuHEDS5cuhY2NjUZxhg0bBoVC8c7f//TTT4JzvHfvHoKDg0vdLBYaMy0tDXv37kVKSopavMDAQI3ifOh8Laad08yZM0u9nqrvPnt7e1SpotkqrbS0NPj5+eHJkyfYt28fEhMTce3aNYwcOVLjOBkZGTh37hxmzJhRqnihpsd2RkYG1q5diydPnmDAgAFqr9mMGTOwefNmjeK9y7Rp05gVGlT57bff0KhRI7Rt21bjv1ulShUEBwczH5BmZWVhx44duHnzplqHCFVngY915syZ9/5e07xZHzfp6elYu3Ytv+xM9TDF0tISnp6eqFu3rkbx3paSkoKUlBQUFhby24S+V9HR0ejTpw/09fURGBiI69evY/bs2YKOGylIcd0p2wFptWrV8Pfff6NevXq4cuUKFi5cyP9OaEsV1aAsJycHOTk5agM+oe7fv890TWVAQAD/T4VCwX9gbt68KSheVlYW+vTpg/Xr1wMoPqGJLftvZGSE7Oxs3L9/n/mFJgtpaWm4evUqPzA5deoUOnXqhOPHj8PS0hIuLi4axZswYQIiIiLg5+eHAQMGwN7ennkVNTE2bdr0wTWyV65c+agTw/nz53Hu3Dm+tY+K0KrC78JiYsaSJUtQv359tfY+c+fOFTwgzcnJgYGBASIjIzFkyBB4enrCxsZG8IB04cKFmDt3Lo4cOQJfX19UrVoVo0ePhrW1teBp/lOnTsWTJ09gYGAAoPjzXa9ePejq6mL9+vWinuiqBAYGYvv27aLjSBVPaMz09HSkp6cjLy8Pd+/e5Y/BrKws5OTkCM5FqVRiwoQJuHTpEtLT07Fq1SqsWLFC4wGpmZkZf84vqWPHjqK+A97G4rN34cIFeHt74/Tp02jYsCE2bNiAqVOnajwgVX22Tp8+jXv37sHBwQFA8WtqYmIiKsc5c+bg22+/hb29veCb2CW5urrCxMQE3bt3FxVP1Qc8IyMDly9f5m+SXrx4kb95JVT9+vXx119/8bO+oqOjUb9+fRw/fhyJiYlq11EfY+HChejTpw/CwsIAAM2bN8e8efM0HpBaWFigb9++yM/P589RYq5vFi9ejMaNG8Pc3Bz79+/HxYsXsXHjRlStWlVw+zQPD49S2y5fvsxv1/TGw7v8+uuvSEhIQMOGDQW1uWvbti2uX7+O9u3bM8kHAHx8fNCiRQskJyfDw8MDhw8fFnRt9/3337/zdwqFQuPBGevjZt68eejcuTNiYmL4mUvp6ekIDw/HvHnzRLUdXLNmDY4cOYJmzZrxN36E7LNKcHAwrKys+JaN48aNw4oVKxAeHq5RnLJuUqmOZTFPnlled/I+dRWlj3Xx4kWuS5cuXLt27bhly5bx28+dO8e5uLgIipmSksI5Ojpybdu25dq1a8c5OTnxlWeFmjJlCpeVlSUqhpQcHR25/Px8ztbWluM4jnv69ClnZ2cnKubp06e5Pn36cP369eM4juOuX7/OTZs2TXSurIwfP57LzMzkf87MzOSmTZvGZWVlcZaWloLjpqenc3v27OHs7e25b775hkWqTIwfP57Jf8NxHBcbG8tt3ryZ69mzJ7d582b+T2hoKPfgwQOxqfJYVFNWHdMsqh9zHMcfGwsWLODOnj1bKrZQhYWF3IkTJzhzc3POwsKC6927N/fzzz8LirVixQru119/5X/+9ddfuXXr1nGXLl3iHBwcROdakf34449cv379OFNTU65fv378n6FDh3IHDx4UHFd1jGzcuJGLiIjgOO5/x6YmvL29OR8fH7XvE9U5liUWnz3V52zNmjX8sSzmszJmzBiuqKiI//nNmzfcmDFjmOTICut4rq6uatcfKSkp3KxZs0TFHDVqFJeXl8f//Pr1a27ChAlcXl6eoO8+1bVCyfd26NChovJjoWQORUVF3JIlS7iJEydyr1+/Fnwc9ujRg1uwYAGnVCo5pVLJHT58mOvatSv/M2slr1E0YWNjw5mYmHCWlpbcsGHD+D9iqI5tVTeCvLw8ztnZWVRMllgdN++7brOwsBAVe9CgQVxOTo6oGCWpvkOCgoK4AwcOqG3ThOr4LflH5dSpU4LzY3ndqSLbJ6TdunXDhQsX8OrVK7U53mZmZnwfR00tWrQIw4cPx7BhwwAU34VdtGgRQkNDBedpYGCAYcOGoXfv3mpTdoWuIQWApKQk3L17FwMHDsSrV69QUFAAQ0NDQbFGjRoFd3d3ZGRkYPPmzThy5Ahmz54tODeg+M7ITz/9xE/fbNeuHVJSUkTFZCk1NRV16tThf65Tpw4ePXoEfX19UdOqVXeZOI6TrLGwEB9z/H7sMd6lSxd06tQJhoaGGDNmjNjU3olFNeW3n/Tn5eWJel+6dOmCb775BhzHYcmSJXj58qXGU9xKevHiBcLDw6FUKtGuXTsEBATg66+/xoMHDzB27FiNexQDxXfsv/vuO/7ngQMHYseOHZgzZw5ev34tONfKwNnZGc7OzggJCRFVFO9tNWrUQHBwMI4dO4b9+/eD4zgUFBRoHMfPzw//+c9/MH78eMyYMQPm5ubvnc4qFIvPXr169bBw4UKcP38eU6dOxZs3b9SmqWkqNTUVeXl5qFGjBgAgPz8fz549E5Vjx44dkZiYiNatW4uKo9KhQwfcunULrVq1YhLv0aNHMDY25n82NjZGUlKSqJgvXrxQOy9WrVoVT548ga6urqDvvrcLAb58+VLUOVY1zU81I0HoLLX8/Hz+3xUKBRYvXgx/f39MnTpV8Ay6Y8eOYenSpUhISMDcuXNRs2ZNbNmyRbMnPBooeY2iCR8fH8aZgD82qlWrhszMTNSpUwdPnz4VFTMrKwtJSUlq74eqC4WmWB031atXx7Vr12BmZqa2/Y8//hC95O7zzz8XPfuwJIVCgaNHj+Lnn39GcHAwAAj6Xnnf8Sum+wTL604V2Q5IgeIWI28vONbT04OVlRVOnz6tcbz09HR+ShBQvH5F0znyb2vWrBmaNWsmKkZJSqUS27dvR0FBAQYOHIjU1FQsW7ZMcPNeW1tbNG7cGP/5z3+Qm5sLf39/dO7cWXSe9evXV/uZxfpZVlq2bAlfX1/Y29tDoVBAqVSiadOmyM/PFzTAiImJQUREBK5evYoBAwbAx8eHyWvIyps3bxAeHo6kpCS0adNG7RgXQtXaR8oBKQtvt/cJDQ0VdYJdvHgxEhMTYWxsDF1dXWRnZ2PFihWC49na2sLe3h5hYWFo1KgRv11oSyiguKjFH3/8wU8Zv3btGr/2UMzguTLIz8+Hrq4unJ2dy1yvKXSNpp+fH8LCwjB//nzUr18fKSkpGDJkiKBY/fr1Q8eOHbF8+XIcP35c1CBPSuvWrcPRo0fh4OCAOnXq4OHDh5gwYYLgeJaWlnBycuJv0qiWV4hx/fp1KJVKNGvWTG2KvNA1pCNGjMCYMWPQqFEjJvEMDQ2xdetWODo6AiiuzSD0xrNKly5d+KnTqgvaTp064dWrV4K+oy0sLLBo0SK8evUKSqUSYWFh/A19IR48eIC5c+fi5s2bUCgUMDExQUBAgNrA/GMYGxvjypUragMcLy8vbNiwQfDygLp16yIwMBBHjx7FmDFjMG/ePEluCInVpUsXAOIHZyU1bdoUmZmZGDJkCJycnGBgYIA2bdoIjhcdHQ1/f3+8fPkSDRo0QEpKClq3bi14nT6r42bp0qWYP38+qlevDiMjIwDFN4by8vLKrD2iCW9vb0yfPh09e/ZU+6wJnYLv6+uLHTt2MGvZyBrr605AxlV238fc3PyDi6fLYmdnh3Xr1qF58+YAip9Ezp07F0qlknWKgtna2mLfvn0YPXo0jhw5AgCwtrZGVFSUqLgsT17jxo3D+vXrMWXKFERERCA2NhZbtmxhWh1YjOzsbGzduhWxsbHgOA5du3blC3C8fPlS44Xr48ePh52dHb755hv+Dr6cfPfdd7h79y46deqE8+fPo3///qJbB8m9tQ/Arr2PaqBS1iAFED5QYdkSSuXq1auYO3cuatSoAYVCgdzcXKxbtw5t2rTByZMnJbubXxHY2dkhIiICrVu3LrVWk+UaTVZ++eUXxMbGYvHixdpORU1hYSHc3NwQEhLCNG5MTAwuX74MjuPQvXt39O3bV1S8y5cvl7lddUGvKSsrKwwbNgwmJiZqa0iFxktNTcXKlSsRGxsLoHhWmI+Pj6gCZQUFBQgPD+dfx65du2LEiBGintwcPXpU7Ryr6TrhkiZMmIDBgwerzVKLiorS+ElKZmYmFApFmU8Z79y5I3gWnUpqaioWLVqEP//8k39/5ILV4Oxdrl69ytceEbpWeujQoQgNDcWkSZNw5MgRnD9/HidPnsTSpUsFxWN13KjEx8fjyZMnAIqfbLZt21b0zYfZs2fj3r17aNWqldrr5ufnp3Gs8tACUorrznI5IO3bt6+gJ6S///47vLy8+Ds/iYmJWLNmjcbFJ9527ty5UtXJ3N3dBcVydHTEoUOHYGtryw9IbWxsEBkZKSheSkoKPD09+cIvLE5e169fx+LFi/Hw4UO0bt0aycnJCA4OlkX1r8LCQvz0009wcnLSdiqfjJWVFY4cOQJdXV1kZWXB2dlZ9E2WytTap6yBipjCCVJW0QSKB9BJSUngOA7NmzeX1eyEyohVhcryZPz48QgNDZXlEySpqM4TlUlZU5STk5PRtGlTQfHKupYRc32TkZHBTy1t1KhRpWixxXpwJgV7e3solUoMGTKEb9E4YsQIjQvyqLA+bkq6cOECevToITqOhYUFTpw4weycyLoFJGtSXHfKdspuWe0NVEqWXNdEnz59EBUVhevXr4PjOHTs2FF0mee1a9ciPj4ed+7cwYABA3Dq1ClBrUVUDA0NkZSUxB/UkZGRatP9NLV48WLm62bbt2+P3bt3448//gBQvK5XLr2cdHR0EBkZyXRAKvcLzurVq/ODEgMDAybrW+Xe2gdg195HdZGpumkjlqqKJktvP8X98ssvARTfgMnNzWXWEoRojlWFyk/VyoKFDh06wM3NDdbW1mqtsIRWlGTdogVgf97u3bs3fv/9d/Tp00dwTiXl5uZi27ZtePDgAZO2VUBxhfk9e/aUeh2FVoi1sbHBN998g4CAAP47Zvbs2YIH5lWqVMG9e/fUZqkJeQqXkpICX19f3LhxAw0aNABQ3GbLxMQES5cuFTxgBoqrm2paif9TkmLpGWu6urrgOA5NmjTBnj17YGRkhIyMDMHxWB03ZY0rFixYgB9++AEcx4l6st60aVPk5OQwaw3IugUka1Jcd8p2QDp16tR3/k7ME4F69eqhZ8+e/NocsRdzZ86cQUREBOzt7bFs2TK4ubkJnpYAFH845s6di6SkJPTv3x81atQQNTVKqpNXQUEBioqKoFAoBN8gkEqPHj3wyy+/MJtuyuqCUyqpqalqg7O3fxZSYEvurX0A9u19WBEyRedDnJycEBERUWZrEDlON61M7t+/j82bN+PUqVOwtraGhYXFe7+/3kWKVhZSUd2M3L9/P79NTIsD1i1aAPbn7YMHD2L79u2oVasWf8EtZtYI67ZVQPGNixYtWohuTaPSqlUrNG3aFOPGjUNISAgMDQ1FXXjOnj0bo0ePRps2bfjzlpC1e/Pnz8eoUaMQGhrKr50vKirCsWPH4OXlhQMHDmgcMzIyEt26dcOZM2f47w45PhVnNTiTkoeHB7Kzs+Hp6YklS5YgKytL1NIDVseNtbU1vvjiC7VtL168wJQpU6BQKHDq1CnBOerr68Pe3p5ZgVPWLSBZk+K6U7YD0piYGOYxT548iRUrVuD58+cAwOQN1tXVRdWqVaFQKFBQUICGDRuKqk7WvHlzHDp0CMnJyeA4Ds2aNRN1spHi5HXy5En4+vqibdu2KCoqQmJiIpYvXy7qi5SlvXv3Mp1uyuqCUyqjRo16789CnDlzBosWLYKOjg5iYmIQHx+PrVu3Ml83JsazZ8+gVCr5dUSurq7w8vJCWFgYhg8frrUBKevm4AD7p7iEHVYVKu/fv49NmzYBAAYNGoRly5Zh2rRpCAoKYpovC6ynkhUVFTGtfAywP2+znjVy+/Zt+Pv749y5cwCAWrVqoaioSFTMly9fYvny5SzSA1B8/TB79mwcOnQIo0aNQnBwsKgpiX369MHPP/+MuLg4UbPUMjMzMXTo0FK52tjY8BVJNfXgwQMcOnQIt2/fhq+vL0xMTGRZuZzV4ExKDRo0gIGBAQwMDPiCnHfv3hUcj9Vx4+7ujri4OCxZsoQvatS/f38m443mzZvz19ksyP27XorrTtkOSP/++288f/681Nzus2fP4vPPPxf0aD0gIAAbN25Ex44dmVWkrFWrFnJzc2FmZgZvb2/Ur19f1IDPw8MDgYGBaNGiRaltQpQ8eQH/WzcrxoYNGxAeHs5XF05OToaLi4tsBqSsLxykKInOktD1yu8j99Y+gHTtfcRi3Rz8benp6YiLi4NCoUCHDh0qxbopOWNVoVKKVhZS4TgOP/30E+7fvw9PT088fPgQz54946s/a4p1ixaA/Xmb9awR1m2rAODf//43UlNTRRVGKkmVj6OjIz7//HNMnDjxncXfPlbdunXRr18/UTEMDQ0RFRWFwYMHq7VjO3bsmODlQ+7u7nB3d4eDgwOGDh2KhIQEvHjxAt9++y3atGmDDRs2iMpZLNXaXVaDMyl5enqWerJc1jZNsDhu3N3dcePGDcydOxc2NjYYOXIkszWfUlyHsWwByZoU+yvbAem6deswa9asUtvr1KmDtWvXCnpSU6dOHcFfmO+yfv166OjowMvLC6GhocjKyhI8eARQ5kX/vXv3BMeT4uRVp04dtVY3TZs2lc2HBAB/54sV1iXRWdu7dy/fouXtNU6bN2/GjBkzBMWVc2sfgH17H1akLESgmp1gamoKjuNkNzuhMlq7di2A4mIj7dq14ytUakqKVhZS8fPzQ1paGhISEuDp6YlatWph1apVgtd8sm7RArA/b7OeNcK6bRVQ/IR06NChMDMzU3sdhV6TlFzH3KtXLwQFBQluQcfS6tWrsXjxYixbtowffKempqJ169ZYvXq1oJiWlpb46quv+JsXY8eORWRkJCIiIpCQkMAyfUHmzJkDpVIJZ2dn7Nq1S/TgTArp6elIT09HXl4e7t69y9/QyMrK4js9aJuJiQl2796NTZs2wdnZWVBvz7KwXnvMugUka1Jcd8q2yu77Wp2UrNz1MVR39Hbv3g0DAwNYWVmpnazlsEj44MGDOHDgAO7du6f2dDQrKwvNmjXT+EvvQ3cxheyzKubOnTuho6MDBwcHcBwHpVIJXV1dTJw4UeOYUnjy5AkCAgKQmJio9mRBzPoAFRYl0Vkruc7l7TUvQtfAyL21D8C+vY8UWDYHB4ovmoKCgkrNTjh+/LjoXIk4+fn5an1DNT3HSt3KgiUbGxscOXIEdnZ2fDV4Tb+XS2LdouVtLM7bw4YNQ0hICKZMmcLvs5WVFaKjowXFY9W2qqR3nesrajuo9PR0tfYdYs/5N2/ehJtaWUFwAAAVh0lEQVSbG0xMTHD79m28ePECkydPRo8ePdCxY0cWKQtmZWUFDw8PrF27FgsXLiz1e7Ezb1jYtWsXdu3ahWfPnvHFpoDiojdjxozhe+7KxbVr13DlyhVRU/lVa489PDz4KsIs1h5L1QKSFSmuO2X7hPR9dy00vaPxdiGQZcuWMVskfO/ePYSEhCAlJUVUdcCePXuiSZMmWL58udpiYH19/VJl1z+Gap/fRcg+v/06lrzrqlAoZDMg9fHxgZWVFW7evIm1a9di//79fFVSMfLz8/mpWvn5+bK4kQFAbZrX2/eXhN5v8vT0xJQpU/Dw4UOMHTuWb+0jF4WFhfj555/h5eVV5u/lMBhl3RwckP/shMqIVW2C972PchqMAsUVFkt+v4hd+8hq4PkunTt3ZhKH5ayRatWqwcXFhelTFVYDz3nz5iEgIADDhg0r8zpCzJNrlurWrcv0XN+mTRvUr18fW7ZsAVA8ADAyMkJ4eLjWB6Rz5szBgQMH8OLFi1LLQlgsBWHB2dkZzs7OCAkJYb4mXApmZmai131Ktfa4WrVqpSr2yuUhCCDNdadsB6R169bFjRs3YGJiorb9xo0bGl+ASbk42MPDAzY2NrCzsxN1sBgZGcHIyIi/+5Geni7qRKva5+DgYFSrVg1OTk7gOA6HDh0S3CRb7ousVTIyMuDo6Ijdu3fDzMwMHTp0wPjx4wXHk6IYFkslLxjevngQuj5Czq19AGna+7AWEhICpVJZqjm4EKrZCb169UJwcLDa7IRBgwaxTJtoSIraBHL31Vdf4ejRo+A4Dg8fPsT27dvRqVMnwfHk3loLKK4X8eLFC/6cGhsbCwMDA8HxsrOzERQUhEuXLkGhUKBr165wdXWFvr6+xrF27doFZ2fnd9aH0LTipbOzMwC884afnIl5Ug8U39BWqV+/PmxsbGBjY8MiNVEGDhyIgQMHws/PDwsWLNB2Ou81ffp03LlzB7GxsQCK+5qXnPmnLYmJifDx8UGVKlXg7+8Pf39/xMbGwtDQECEhIYKm9Eu19ph1C0jWpLjulO2A1NXVFa6urnBzc0O7du0AAPHx8QgKChLVVoW1qlWrYvLkycziXb9+HR4eHigqKsKZM2cQHx+PgwcPCq6c9/vvv6uV5p80aRJGjhyJcePGsUpZdlQDbj09PTx+/Bj/+te/8PjxY8Hx5H7BWbLcdsl/5zgOz549ExxXzq19APbtfVirWrUq6tWrx0/j7Nmzp+BekuVldkJlJEVtArnz9vbG6tWr8fz5cwwfPhz9+/eHt7e34Hhyb60FsJ814uPjA319fSxcuBAcxyEiIgI+Pj58pWVNqJYglexXKEbbtm0BqD+5zs/Pxz///FPqKbE2vK9PvZh+l0Bxj10VMf3apSL3wSgAHDlyBGvXrkXfvn0BANu2bYOnp2epysif2ooVK+Dm5oasrCxMnjwZs2fPxvbt2xETEwN/f39B6zOlWnvMugUka1Jcd8p2QNq7d2+sWLECQUFBWLlyJQDA1NQUy5YtQ+/evbWc3f+wbpa9atUq7NixA56engCKq5uK+aLPzMzE/fv30aRJEwDFRZMyMzOZ5CpXnTt3RmZmJkaOHAl7e3vo6uqKGrTI/YKzZLntt0tvjxw5UlBMubf2Adi392GNZXPw8jI7oTJRPbUeNGgQwsLCZFmbQCr6+vpYsWIFs3hyb60FsJ81cu/ePbX1YJ06dYK1tbWgWCNGjADAvvLl7NmzsWzZMlSrVg02NjbIyMjAtGnTMGnSJKb/H02pptKWNTVQ7PXNmTNnZDH9tTz74YcfEBERwd+8eP78OSZNmqT1AemrV68wYMAAAMU3dVX59O/fX9CNIAA4fvw4v/Z406ZN/NrjoKCgUl1CNMG6BSRrUlx3ynZAChRPT+vVq5e203iv7t27w9XVFVWqVGHSLLugoKDUeiGhU2yB4i+U4cOH83c8b9y4wbRPmRypphnZ2tqiS5cuyM7OxldffaVxnPJywSlF+W25t/YB2Lf3YY11c3AiL1LWJpC7b7/9FmPGjIGdnV2pdU5CyL21loqBgQGzwcoXX3yhtjQnIyMDjRs3FhTrQ63chDSpB4rbThgYGOCXX35B165dsWDBAgwfPlzrA1IjIyOEhYWV2d5G6Pvz9OlTNGrUCMHBwXwMX1/fCn+9JJWST9Ll8FQdUF/b2LNnT7XfiVkHL8XaY9YtIFmrVG1fLl68iO7du4v+b6S2aNEi+Pn5wdTUlMl0Tl1dXbx69Yqfg33nzh21QZCmLCws0LlzZ/z555/gOA5mZmayKPgihbIqC3/22Wf47LPPkJubq/EAsrxccN65c0etuEN0dDSioqJgbGyMGTNmCFqTVB6K57Bu78Oa6txUsjk4qTgq81Pr9evXY9++fdi6dSssLCwwatQoQcX3VOTeWosl1eBRT08PNjY2fPuO06dPC76eUU3VTUlJwZUrV/h15b/99puom/qqpRpXrlyBubk5atasKYtlKxYWFnj06FGZA1Kha+q9vLyQnp6OjIwMHDt2DKampoiPjxebaqX05ZdfYtOmTXBycoJCocDBgwdhbGys7bT4XsJvz/B4+vSp6AcMrNces24ByZoU152ybftiY2ODAwcOvLda08iRI/lyyNri4ODAtOLcmTNnEBwcjAcPHqB37944e/YsAgICRD36ryxat26tNoAEIMsBJGsODg4ICgpCgwYNEB8fD2dnZ7i4uODWrVuoWrWqRn3ZyktrH0Da9j4spKWlwc/PD0+ePMG+ffuQmJiIa9euCZ7OQojcvHz5EocPH8aPP/6Ixo0bw9nZGRYWFqJiyrG1FkuqpyjvIubJw5QpU7BmzRp89tlnAIqfunp7e2Pbtm2C4nl4eCArKwv37t1DdHQ0qlSpAicnJ0RGRgrOUc5ev34Ne3t7WFlZISEhARcuXIC5uTm6d+9O520NpKWlYcWKFbhw4QIUCgV69OiB7777DvXq1dN2amXKyclBbm6uLPJj3QJSKiyvO1Vk+4T01q1bMDMzK3NAqhpk/Otf/9JCZuoGDhyI/fv3w9LSksl0TnNzczRv3hxnz54Fx3FwcXHh13+S96usTyzy8vL4nl8nTpyAnZ0dpkyZgsLCQo3v0JWn4jlStfdhZeHChejTpw/CwsIAFK8JmTdvHl3YkAojLi4OsbGxqFGjBnr37o3w8HBER0dj48aNgmOyatEiV1JMdVN58uQJPxgFimcIPXr0SHA8f39/nDt3Dq1atYKenh5SU1Mxd+5cFqnKzsyZM9G1a1fo6ury75GNjQ1cXV1lU5egvKhevXqp6rLZ2dlayubD9PT0RBcEY7X2mHULSKmwvO5Uke2AtLwMLlRfvEuXLmX2NM7Y2LjUImFC3qXkFKq4uDiMHj0aQHFrFE2nV5WXzx3Avr0Pa6mpqRg5ciQOHDgAoHg6vhymuxEi1g8//IDw8HAYGxtj7NixMDc3h0KhwPTp06kN0UcKDg5m2oMUKL7p9d1338HBwQEAoFQqRfVZrFGjhlrdgIYNG5Y5TbYicHNzw6VLl/D48WNYWFjA2NgYaWlpePXqVYXuSiCFsWPHluq3Xda2ioD12mPWLSClwvK6U0W2A9LygvUF/H//+1+sW7cOKSkpKCwslF3lUCI/n3/+Ofbt24eGDRsiISGBX4eUn58vy3YtrLBu78Na1arqp9eXL18KbhhNiJw8ePAAwcHBZfYWFNp3r7KIjIxEt27dcObMGX5Aamdnx+RifdWqVdi6dSuWL18OjuPQrVu3ctlLVBtatWqFVq1aITo6GgcOHEBycjImTpwIpVKJxYsXq1VEJmV78+YN3y7u9evX/PddVlZWmTU+KgKp1h6zbgHJmhTXnTQglRkfHx/MmjULbdu2pacp5KMsWrQIS5cuRWpqKpYuXYo6deoAKC76peoDVhGxbu/DmoWFBRYtWoRXr15BqVQiLCwMw4YN03ZahIj2vmrRqorupGwPHjzAoUOHcPv2bfj6+sLExASvX79mEltfX58GoCKpphs2bdoUtWvX5tsOkg8LCQnBli1boFAo1KrL6uvrY8KECVrMTDq7du3i1x7fv38f0dHRSEpKwsyZM0WtPWbdApI1Ka47ZVvUqLJydHTEoUOHtJ0GIeXK48ePBbf3kdLRo0cRExMDjuPQv39/UVX3CJELuRcUKw8cHBzg5eWFhIQEbN26FfXq1UObNm1EPWGWYhpwZfbo0SPZV3OXo2XLlmHRokXaTuOTUK09PnToEF9k1cbGBv7+/rh48aLggfiwYcNw+PBh2Nra8nFL/ntFRI/gZMba2hr79+9HZmYmcnNz+T+EvMvHTOeuSFO+S34uVH8+++wzGBsby+azUlhYiE2bNmHo0KHYuHEjAgMDaTBKKgwfHx9+itbatWvRqVMn2NnZaTmr8sHS0hIeHh58v9WxY8eicePG+OWXXzB58mRBMSMjI5GamoozZ87w2+j9EI8Go8JUlsEoULz2+M2bN/za40mTJjFZe8y6BSRrUlx30hNSmYmKioKvry8/haeitywh4pWXFkmslJf2PmPHjsWePXu0nQYhzKnu1A8ZMgTHjh1DUVERxo8fj927d2s7tXLh5s2bcHNzg4mJCW7fvo0XL15g8uTJ6NGjh9pUx4+1ZcsWXLp0CTdu3MDgwYNhYmKC3bt34/jx4xJkTwh5m5OTk9ra4+7duyMuLk7w2mO5t4CU4rqT1pDKzPr167F7926YmprSGlLyUcpLiyRWyksl4L59+2Lnzp2wtbVVKykvtgE3Idom94JictemTRvUr1+f70lqbW0NIyMjhIeHCxqQuru7w93dHQ4ODhg6dCgSEhLw4sULfPvtt6KnARNCPoz12mO5t4CU4rqTnpDKzIgRIxAeHq7tNAghIrVu3Zr/dzk+wSVEKH9/f0ybNg2nT5/G6tWr+YJiPj4+2k6t3IiLi0OHDh0AABMmTEBoaKjgWJaWlvjqq6+QkJCAoKAgtGjRAg4ODoiIiEBCQgJMTU1ZpU0I+QBaeywMDUhlZuPGjSgoKICVlZXafPGWLVtqMStCCCGkNLkWFKtsWE8DJoRoT2VsAUkDUpnp379/qW0KhYKqFxJCCNGqDxUNo+noH+/MmTMwNzdnGlO1jg0ongY8ZcoUXLx4EatXr2b6/yGESOubb74pswVkRX7ySmtIZSYmJkbbKRBCCCGlmJmZlYuCYnL29OlTNGrUCMHBwfyA1NfXl0nD+5JTpuvXrw8bGxuq7k1IOVS7dm1YWlpqO41Pip6QylRaWppaf7cvvvhCi9kQQgghRCxnZ2ekp6cjIyMDXl5eMDU1xZw5cypMFXRCiHi7du2Crq4uLC0t1ZbvVeRZKDQglZmLFy/C29sbaWlpqFKlCgoKCmBoaFih540TQgghlcXr169hb28PKysrJCQk4MKFCzA3N0f37t0xcuRIwXGlmAZMCPn0KmMLSOorIjMBAQH48ccf0bJlS8TFxWHp0qVwcnLSdlqEEEIIEWnmzJk4fPgwdHV14e7ujuDgYDRt2hSurq78xaemnj59CgAIDg7mt/n6+jLJlxDy6alaQCYkJODmzZtITEys0INRgNaQylKzZs3w5s0bKBQKDB8+HKNHj9Z2SoQQQggRyc3NDZcuXcLjx49hYWEBY2NjpKWl4dWrVxg3bpygmF5eXvw04GPHjsHU1BTx8fGMMyeEfCoNGjRAu3bttJ3GJ0VPSGWmatXiewQNGzZETEwMbt26xd/9JIQQQkj51apVKzg7O6NZs2Y4efIkfH19oaurC6VSKbgA0a5du3Do0CHUrl0b9+/fR0BAAJKSkjBz5kzs37+f8R4QQqTWrVs3BAQEICEhAXfu3OH/VGT0hFRmxo0bh3/++QceHh6YO3cusrKyqNk4IYQQUoGoBp9NmzZF7dq1sXLlSsGxZs6cia5du/LTgFXxXV1dqf4EIeXQ0aNHAQDHjx/nt1X0FpBU1EhGioqKcPbsWSpKQAghhFQSjx49EtVf8NatW7h06RK2bt0KQ0NDGBsb49atWwgMDETHjh2ho6PDMFtCCGGPBqQyM2LECISHh2s7DUIIIYSUI05OTjhw4ACSk5MxceJEdO/eHXFxcYiKitJ2aoQQASpTC0iasiszbdu2xfXr19G+fXttp0IIIYSQcoLlNGBCiPZUxhaQ9IRUZmxtbfH333+jSZMm0NPT47f/9NNPWsyKEEIIIeWF2GnAhBDtsbe3x7p16zB79mxERETg0KFDePz4MWbNmqXt1CRDT0hlhgoYEUIIIUQMGowSUr5VthaQNCCVmS5dumg7BUIIIYQQQogWvN0C0sjIqMK3gKQBqcxkZWVhx44duHnzptpC5t27d2sxK0IIIYQQQojUKmMLyCraToCo8/HxQZUqVZCcnIzhw4dDR0eHChwRQgghhBBSwRUVFcHAwAB16tRB+/bt8euvv+LSpUsYOnSotlOTFA1IZeb+/fuYNWsWatSoAWtra2zbtg1//fWXttMihBBCCCGESKhKlSoIDg7WdhqfHA1IZUZXVxcAUK1aNWRmZqJatWoVft44IYQQQggh5H8tICsTWkMqM02bNkVmZiaGDBkCJycnGBgYoE2bNtpOixBCCCGEECKxq1evYv/+/ZWqBST1IZWxq1evIisrC3369IGOjo620yGEEEIIIYRI6PLly2Vur8idOGhAKkPp6emIi4uDQqFA+/btUbduXW2nRAghhBBCCCHM0ZRdmTl58iR8fX3Rtm1bFBUVITExEcuXL8fAgQO1nRohhBBCCCFEQpWxBSQNSGVmw4YNCA8PR7NmzQAAycnJcHFxoQEpIYQQQgghFZyPjw9atGiB5ORkeHh44PDhwzA1NdV2WpKiKrsyU6dOHX4wChQXOTI0NNRiRoQQQgghhJBPoTK2gKQBqcz06tULwcHBeP78OZ49e4aQkBAMGjQIubm5yM3N1XZ6hBBCCCGEEIlUxhaQNGVXZrZs2QIACAwMVNu+Zs0aKBQK3Lx5UxtpEUIIIYQQQiRWGVtAUpVdQgghhBBCCJGZytICkp6QEkIIIYQQQohMlGwB2aFDhwo9GAVoDSkhhBBCCCGEyMLJkydhaWmJvXv3YteuXRg8eDB+++03baclKZqySwghhBBCCCEyYGlpiaCgoFItII8fP67lzKRDT0gJIYQQQgghRAYqYwtIekJKCCGEEEIIITKwZcsW6OjowMHBARzHQalUQldXFyNHjgQA1KxZU8sZskcDUkIIIYQQQgiRgdatW7/zdxW1BSQNSAkhhBBCCCGEaAWtISWEEEIIIYQQohU0ICWEEEIIIYQQohU0ICWEEEIIIYQQohU0ICWEEEIIIYQQohU0ICWEEEIIIYQQohU0ICWEEEIIIYQQohX/D3hp2RR5x70CAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "ax = sns.barplot(x=token_labels, y=e_scores, ci=None)\n",
    "ax.set_xticklabels(ax.get_xticklabels(), rotation=90, ha='center')\n",
    "ax.grid(True)\n",
    "plt.title(\"Start Word Scores\")\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-06T07:44:15.727880Z",
     "start_time": "2020-05-06T07:44:14.765354Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABvsAAAHzCAYAAAAO46E8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzde5CddWH/8c9udoOQBCHhoMQgoCimRUQRcIoKWClIQjMpWpDgaEeItBVaOzZSFLmIxIQwjIxVdBCdiUnjaAUFFaQ1XmqQm3iriqKAhkiIaSAbYG/Z/f3BmDY/LjmcPSfP99l9vf5iL8/Z9zl7dhPyOec5XaOjo6MBAAAAAAAAaqe76gAAAAAAAACgNcY+AAAAAAAAqCljHwAAAAAAANSUsQ8AAAAAAABqytgHAAAAAAAANdVTdcDOsnHjloyMjI75cvbcc7ds2vRYG4raR1PzSuwqsSkps0tT80rsKrEpKbNLU/NK7NLUvBK7SmxKyuzS1LwSu0psSsrs0tS8Ers0Na/ErhKbkjK7NDWvxK4Sm5IyuzQ1r8QuTc0rsaudTY3GtLZcDv/LM/uepZ6eSVUnPImm5pXYVWJTUmaXpuaV2FViU1Jml6bmldilqXkldpXYlJTZpal5JXaV2JSU2aWpeSV2aWpeiV0lNiVldmlqXoldJTYlZXZpal6JXZqaV2JXiU38L2MfAAAAAAAA1JSxDwAAAAAAAGrK2AcAAAAAAAA1ZewDAAAAAACAmjL2AQAAAAAAQE0Z+wAAAAAAAKCmjH0AAAAAAABQU8Y+AAAAAAAAqCljHwAAAAAAANSUsQ8AAAAAAABqytgHAAAAAAAANWXsAwAAAAAAgJoy9gEAAAAAAEBNGfsAAAAAAACgpox9AAAAAAAAUFM9VQcAAAAAAADbm7b7c/KcXXpbOnZwaGuba4CSGfsAAAAAAKAwz9mlN6ctWtHSsSuXLmhzDVAyp/EEAAAAAACAmjL2AQAAAAAAQE0Z+wAAAAAAAKCmavOafQMDA7n00ktzyy23ZJdddsmhhx6aD33oQ1VnAQAAAAAAQGVqM/Zddtll2WWXXXLTTTelq6srf/jDH6pOAgAAAAAAgErVYux79NFHc9111+Xb3/52urq6kiR77bVXxVUAAAAAAABQrVq8Zt/vfve77LHHHvnYxz6Wv/qrv8rb3va23HHHHVVnAQAAAAAAQKW6RkdHR6uO2JGf/vSnOfnkk7Ns2bKcdNJJ+dGPfpSzzjorN998c6ZOnVp1HgAAAAAAtN1pi1a0dNzKpQvaXAKUrBan8Zw5c2Z6enoyd+7cJMkrXvGK7Lnnnrn33nvz8pe/vKnL2LhxS0ZGxr5rNhrTsmFD35gvp500Na/ErhKbkjK7NDWvxK4Sm5IyuzQ1r8QuTc0rsavEpqTMLk3NK7GrxKakzC5NzSuxS1PzSuwqsSkps0tT80rsKrEpKbNrojU1GtPGdPxEuq1aVWJTUmZXO5vGet/myWpxGs/p06fnyCOPzPe+970kyb333puNGzdmv/32q7gMAAAAAAAAqlOLZ/YlyUUXXZTzzjsvS5YsSU9PT5YuXZrdd9+96iwAAAAAAACoTG3Gvn333TfLly+vOgMAAAAAAACKUYvTeAIAAAAAAABPZuwDAAAAAACAmjL2AQAAAAAAQE0Z+wAAAAAAAKCmjH0AAAAAAABQU8Y+AAAAAAAAqCljHwAAAAAAANSUsQ8AAAAAAABqytgHAAAAAAAANWXsAwAAAAAAgJoy9gEAAAAAAEBNGfsAAAAAAACgpnqqDgAAAAAAAMa3keGhNBrTWjp2eHAgmx4ZbHMRjB/GPgAAAAAAoKO6e3pz59IzWjr2sEVXJzH2wdNxGk8AAAAAAACoKWMfAAAAAAAA1JSxDwAAAAAAAGrK2AcAAAAAAAA1ZewDAAAAAACAmjL2AQAAAAAAQE0Z+wAAAAAAAKCmjH0AAAAAAABQU8Y+AAAAAAAAqCljHwAAAAAAANSUsQ8AAAAAAABqytgHAAAAAAAANWXsAwAAAAAAgJoy9gEAAAAAAEBNGfsAAAAAAACgpox9AAAAAAAAUFPGPgAAAAAAAKgpYx8AAAAAAADUlLEPAAAAAAAAasrYBwAAAAAAADVVu7HvYx/7WA466KD88pe/rDoFAAAAAAAAKlWrse+///u/88Mf/jAzZ86sOgUAAAAAAAAqV5uxb3BwMBdffHEuuOCCdHV1VZ0DAAAAAAAAlesaHR0drTqiGZdddllmzpyZBQsW5A1veEOuuuqqvPSlL606CwAAAAAAOuK0RStaOm7l0gVtLmmPO5ee0dJxhy26us0lML70VB3QjLvuuis/+clP8t73vrfly9i4cUtGRsa+azYa07JhQ9+YL6edNDWvxK4Sm5IyuzQ1r8SuEpuSMrs0Na/ELk3NK7GrxKakzC5NzSuxq8SmpMwuTc0rsUtT80rsKrEpKbNLU/NK7CqxKSmza6I1NRrTxnR8ibfVWHTi+pR4n0rK7Gpn01jvCzxZLca+22+/Pb/5zW/y53/+50mSBx98MO985zuzePHivPa1r624DgAAAAAAqJuR4aGWh6fhwYFsemSwzUXQmlqMfQsXLszChQu3ve00ngAAAAAA8NSMWM3p7ukd46lFJ8btRPlqMfYBAAAAAADNMWLBxFLLse+b3/xm1QkAAAAAAABQue6qAwAAAAAAAIDWGPsAAAAAAACgpox9AAAAAAAAUFPGPgAAAAAAAKgpYx8AAAAAAADUlLEPAAAAAAAAasrYBwAAAAAAADVl7AMAAAAAAICaMvYBAAAAAABATRn7AAAAAAAAoKaMfQAAAAAAAFBTxj4AAAAAAACoKWMfAAAAAAAA1JSxDwAAAAAAAGrK2AcAAAAAAAA1ZewDAAAAAACAmjL2AQAAAAAAQE0Z+wAAAAAAAKCmjH0AAAAAAABQU8Y+AAAAAAAAqCljHwAAAAAAANSUsQ8AAAAAAABqytgHAAAAAAAANWXsAwAAAAAAgJoy9gEAAAAAAEBNGfsAAAAAAACgpox9AAAAAAAAUFPGPgAAAAAAAKgpYx8AAAAAAADUlLEPAAAAAAAAasrYBwAAAAAAADVl7AMAAAAAAICaMvYBAAAAAABATRn7AAAAAAAAoKZ6qg5oxqZNm7Jo0aL89re/zeTJk7Pffvvl4osvzvTp06tOAwAAAAAAgMrU4pl9XV1dOeOMM3LTTTfl+uuvz7777ptly5ZVnQUAAAAAAACVqsXYt8cee+TII4/c9vahhx6adevWVVgEAAAAAAAA1avFaTz/r5GRkfzbv/1b3vCGNzyr42bMmNq2hkZjWtsuq100Na/ErhKbkjK7NDWvxK4Sm5IyuzQ1r8QuTc0rsavEpqTMLk3NK7GrxKakzC5NzSuxS1PzSuwqsSkps0tT80rsKrEpKbNL085R4nWaaE0T7foyNrUb+z70oQ9lt912y+mnn/6sjtu4cUtGRkbH/PUbjWnZsKFvzJfTTpqaV2JXiU1JmV2amldiV4lNSZldmppXYpem5pXYVWJTUmaXpuaV2FViU1Jml6bmldilqXkldpXYlJTZpal5JXaV2JSU2TXRmqocVjpxncZ6fSZKUzL+7+tGw/ar1di3ZMmS3H///bnqqqvS3V2LM5ACAAAAAABAx9Rm7Lviiivy05/+NJ/61KcyefLkqnMAAAAAAACgcrUY+371q1/lqquuyv77759TTz01STJr1qz867/+a8VlAAAAAAAAUJ1ajH0veclLcvfdd1edAQAAAAAAAEXxwncAAAAAAABQU8Y+AAAAAAAAqCljHwAAAAAAANSUsQ8AAAAAAABqytgHAAAAAAAANWXsAwAAAAAAgJoy9gEAAAAAAEBNGfsAAAAAAACgpox9AAAAAAAAUFPGPgAAAAAAAKgpYx8AAAAAAADUlLEPAAAAAAAAasrYBwAAAAAAADXVU3UAAAAAAAAATxgZHkqjMa2lY4cHB7LpkcE2F1E6Yx8AAAAAAEAhunt6c+fSM1o69rBFVycx9k00TuMJAAAAAAAANeWZfQAAAABAWzkFHQDsPMY+AAAAAKCtnIIOAHYep/EEAAAAAACAmjL2AQAAAAAAQE0Z+wAAAAAAAKCmjH0AAAAAAABQU8Y+AAAAAAAAqCljHwAAAAAAANSUsQ8AAAAAAABqytgHAAAAAAAANWXsAwAAAAAAgJoy9gEAAAAAAEBNGfsAAAAAAACgpox9AAAAAAAAUFPGPgAAAAAAAKgpYx8AAAAAAADUlLEPAAAAAAAAasrYBwAAAAAAADVVm7Hv3nvvzSmnnJLjjz8+p5xySu67776qkwAAAAAAAKBStRn7Lrjggpx22mm56aabctppp+WDH/xg1UkAAAAAAABQqVqMfRs3bszPfvazzJ07N0kyd+7c/OxnP8v//M//VFwGAAAAAAAA1ekaHR0drTpiR37605/mfe97X7761a9ue9+JJ56Yyy67LH/6p3/6rC9vcGhrJvdOaqll69BgJvVObunYkeGhdPf0ahpDU6ldJTaV2qWpuaZSu0psqqqrxKYddZXYVGqXpuaaSu0qsanULk3NNZXaVWJTVV0lNu2oq8SmUrs0NddUaleJTaV2aWquqdSuEpuq6iqxaUddJTaV2lVi01i6SmzqZFeJTbRHT9UBO8vGjVsyMvLErtloTMtpi1a0dDkrly7InUvPaOnYwxZdnQ0b+p7yY5qaayq1q8SmUrs0NddUaleJTVV1ldi0o64Sm0rt0tRcU6ldJTaV2qWpuaZSu0psqqqrxKYddZXYVGqXpuaaSu0qsanUrvHY9Ezfgx3rf9qPjMfbajzd10ts2lFXiU1j1WhM68jP4Hj7vVBiU9K5rlLu643GtJYuh6dXi7Fvn332yfr167N169ZMmjQpW7duzUMPPZR99tmn6jQAAAAAGLP+gaGsXLqg6gwAoIZqMfbNmDEjs2fPzg033JB58+blhhtuyOzZszN9+vSq0wAAAABgzPo296fvGZ4h8kw8QwIAJrZajH1JcuGFF+bcc8/Nxz/+8ey+++5ZsmRJ1UkAAAAAAABQqdqMfS9+8YvzhS98oeoMAAAAAAAAKEZtxj4AAADaY2R4KIcturqlY4cHB9pcAwAAwFgY+wAAACaY7p7ebNjQV3UGANBh/QNDWbl0QdUZAHSYsQ8AAAAAYBzq29yfvvS3dGyjMa3NNQB0SnfVAQAAAAAAAEBrjH0AAAAAAABQU07jCQAAAADATuF1BAHaz9gHAAAAAMBO4XUEAdrPaTwBAAAAAACgpox9AAAAAAAAUFPGPgAAAAAAAKgpYx8AAAAAAADUVE/VAQAAAFCqkeGhHLbo6paOHR4caHMNAADAkzU99o2OjuYLX/hCbrjhhmzatCnXX399br/99mzYsCEnnnhiJxsBAAD4//QPDGXl0gUtHTs4tLXNNeNXd09vNmzoqzoDAADgaTU99n30ox/NmjVr8va3vz0XXHBBkuT5z39+Fi9ebOwDAADYyfo296cv/S0d22hMa3MNAAAAVWn6NfuuvfbaXHXVVZkzZ066urqSJLNmzcrvfve7jsUBAAAAAAAAT6/psW/r1q2ZMmVKkmwb+x599NHstttunSkDAAAAAAAAnlHTp/F8/etfn8WLF+e8885L8sRr+H30ox/Nscce27E4AAAAAIB2GRkeymGLrm7p2OHBgTbXAEB7ND32nXfeeVm0aFEOO+ywDA8P55WvfGWOOuqoLFmypJN9AAAAAABt0d3Tmw0b+qrOAIC2amrsGx0dzaZNm3LllVfmkUceyQMPPJB99tknjUaj030AAAAAAADA02jqNfu6urpy0kknpbu7OzNmzMghhxxi6AMAAAAAAICKNTX2Jcns2bNz7733drIFAAAAAAAAeBaafs2+I444ImeeeWbmz5+f5z//+enq6tr2sTe/+c0diQMAAAAAAACeXtNj3w9+8IO84AUvyG233bbd+7u6uox9AAAAAAAAUIGmx77ly5d3sgMAAAAAAAB4lpoe+5LkkUceyerVq7N+/fo873nPy7HHHpvnPve5nWoDAAAAAAAAnkF3s59411135bjjjsuqVaty9913Z9WqVTnuuONy1113dbIPAAAAAAAAeBpNP7Pv0ksvzQUXXJA5c+Zse9/Xvva1XHLJJfn3f//3jsQBAAAAAAAAT6/pZ/bdd999edOb3rTd+44//vj89re/bXsUAAAAAAAAsGNNj3377bdfvvrVr273vhtvvDH77rtv26MAAAAAAACAHWv6NJ7nnXdezjrrrCxfvjwzZ87MAw88kPvvvz9XXXVVJ/sAAAAAAACAp9H02PeqV70qN998c771rW/loYceyrHHHpujjz46e+yxRyf7ijMyPJTDFl3d0rHDgwNtrgFgrFr9ve53OgAAAAAThW2kbE2PfevXr89znvOczJs3b9v7Hnnkkaxfvz7Pe97zOhJXou6e3mzY0Fd1BgBt4vc6AAAAADwz/4ZWtqZfs+/v/u7v8uCDD273vgcffDDvfve72x4FAAAAAAAA7FjTY999992Xgw46aLv3HXTQQfnNb37T9igAAAAAAABgx5o+jef06dNz//33Z7/99tv2vvvvv7/jr9l30UUX5ZZbbsnkyZOz22675f3vf39e/vKXd/RrAgAAAAAAQB00/cy+k08+OWeffXZWr16de+65J9/85jdz9tln5y1veUsn+/L6178+119/fb7yla/kXe96V97znvd09OsBAAAAAABAXTT9zL6FCxemp6cnS5YsyYMPPph99tknb3nLW/KOd7yjg3nJscceu+2/Dz300Dz44IMZGRlJd3fTOyUAAAAAAACMS02PfbfddluOP/74nHHGGXnooYeybNmy/OpXv8rGjRvTaDQ62bjNihUrcswxx7Q09M2YMbVtHY3GtLZdVglKvD4lNiVldpXYlJTZpal5JXZ1sqnE6zsWJV6fEpuSMrs0Na/ErhKbkjK7NDWvU12lXt9W+btC80q9PiV2aWpeiV0lNiVldk20P2tK7WpVidenxKakzK6J9ncYTc0rsavEJp7Q9Nh30UUX5dOf/nSSZMmSJU8c3NOT888/P1dddVXLAfPnz8+6deue8mNr1qzJpEmTkiRf/epXc/3112fFihUtfZ2NG7dkZGQ0ydjvkBs29I3p+KdS5Q/J012fEpuSMrtKbErK7NK0Pd+/5nXytvJ7vfPc15tXp6akzK4Sm5IyuzRtr27fv7FoNKZ15HL/eNlV8HeF5tXtvq5pe75/zavbbTUWnfo9V+Lvz6TMrvF2n0rGdp1KbErq97tqLDr1970Sf/6SiXV/HYtSmoyG7df02Ld+/frMnDkzw8PD+e53v5vVq1ent7c3r3vd68YUcO211+7wc26++eZcccUV+exnP5u99tprTF8PAAAAAAAAxoumx76pU6fmD3/4Q371q1/lwAMPzJQpUzI4OJjh4eFO9mX16tVZvHhxPvOZz2TWrFkd/VoAAAAAAABQJ02Pfaeffnre/OY3Z2hoKOedd16S5Ac/+EFe9KIXdSwuSf7lX/4lvb29Oeecc7a977Of/Wz23HPPjn5dAAAAAAAAKF3TY9/ChQtz3HHHZdKkSXnhC1+YJHne856XSy65pGNxSfL973+/o5cPAAAAAAAAddX02JckBxxwwDO+DQAAAAAAAOw83VUHAAAAAAAAAK15Vs/sAwAAAABge/0DQ1m5dEHVGQBMUMY+AAAAAIAx6Nvcn770t3RsozGtzTUATDRO4wkAAAAAAAA1ZewDAAAAAACAmjL2AQAAAAAAQE0Z+wAAAAAAAKCmjH0AAAAAAABQUz1VBwAAAPxR/8BQVi5dUHUGAAAA1IaxDwAAKEbf5v70pb+lYxuNaW2uAQAAgPIZ+wAAAAAAmLCcXaJ5Y7mtBoe2trkG+CNjHwAAAAAAE5azSzTPbQVl6q46AAAAAAAAAGiNsQ8AAAAAAABqytgHAAAAAAAANWXsAwAAAAAAgJoy9gEAAAAAAEBNGfsAAAAAAACgpox9AAAAAAAAUFPGPgAAAAAAAGjS2972tvzwhz+sOmMbYx8AAAAAAAA0YXh4uNLjn0pP2y8RAEj/wFBWLl1QdQYAAAAATHhr167NO9/5zhx55JG544478vznPz//9E//lCVLlmTdunU588wzc+qpp+acc87J2rVrMzAwkL/8y7/Mu971riTJG97whsyfPz+33HJL5s+fv+1yR0dH8+EPfzgjIyM5//zzc/fdd2fx4sXp6+vLrrvumosvvjgvfvGLc+6552batGm5++67s//+++fiiy9u6/Uz9gFAB/Rt7k9f+ls6ttGY1uaa8WtkeCiHLbq6pWOHBwfaXAMAAABAqX7729/m8ssvz8UXX5yzzjorS5Ysyac//ek8/PDDOemkk3LKKafk4osvzh577JHBwcG89a1vzV/8xV/kgAMOSJL09PRk5cqVSZKvfOUrGRkZyfvf//5Mnz49733vezM0NJQLL7wwV155Zfbee+/cddddueCCC/K5z30uSXLvvffmmmuuSU9P+6c5Yx8AUFvdPb3ZsKGv6gwAAAAACrfPPvvk4IMPTpLMnj07kyZNyuTJk7P33ntn1113zaZNm7Jq1arcdNNNGRkZyfr16/PrX/9629g3Z86c7S7voosuyjHHHJP3vOc9SZ4Y8375y1/mzDPP3PY5jz/++Lb/Pv744zsy9CXGPgAAAAAAAMa5yZMnb/vv7u7uJ719zz33ZPXq1Vm5cmWmTJmSs88+OwMD/3tmqF133XW7y3vVq16V22+/PY8++mimTJmS0dHR7L///vnSl770lF9/t912a/M1+l/dHbtkAAAAAAAAqIF169Zl9913z5QpU7Ju3bqsWbPmGT9/3rx5mTdvXv72b/82/f39edGLXpS+vr7ceuutSZKRkZH8/Oc/3xnpntkHAAAAAED5vG470ElHHHFEvv71r+ekk07KC1/4whx++OE7POaUU05Jf39//v7v/z6f+MQn8rGPfSyXXHJJPvzhD2doaChvetObMnv27I63G/sAAAAAACie120HWjVr1qzceOON294+++yzt/v4N7/5zSTJJz/5yac8/o8f/6Ply5dv+++3v/3tefvb354kOeigg7b72B995CMfaS28SU7jCQAAAAAAADXlmX0AAAAAANACpxYFSmDsAwAAAACAFji1KFACp/EEAAAAAACAmjL2AQAAAAAAQE3VZuy79dZbM3v27Hzuc5+rOgUAAAAAAACKUIvX7NuyZUuWLVuW17/+9VWnAAAAAEBRRoaHctiiq1s6dnhwoM01ALBzDA5tzeTeSbW53E6qxdj3kY98JO985zvzrW99q+oUAAAAAChKd09vNmzoqzoDAHaqyb2TctqiFW2/3JVLF7R87Nq1a/O9730vp5xySkvH33rrrRkaGsprX/vaZ3Vc8WPft7/97WzevDknnHDCmMa+GTOmtq2p0ZjWtssqQYnXp8SmpMyuEpuSMrs0Na/Erk42lXh9x6LE6+P717wSr0+JTUlnusbyqPiR4aEJdVuNlabmdaqrxOvb6s9gp3/+SrytxqLU61Nil6bmldhVYlMysX6vl9iUlNvVqhKvz0T7c7nEpqTMLk3NK7GrxKbSPPDAA/n85z/f0tg3PDyc2267LY899lj9xr758+dn3bp1T/mxG2+8MZdffnk+85nPjPnrbNy4JSMjo0nGfofsxCOlqvwhebrrU2JTUmZXiU1JmV2atuf717xO3lalPQK2U9fH9695Jd5WJTYlZXaNpWnsj4rvf9qPjLfbaqxK/Nmt2319LBqNaR3786+6n8HO/fyVeH8di7rd1zVtz/eveXW7rcaiU7/XS/z9mZTb1apSr89YujrZVOL3r7SmpMyuUv8OWmJTUubfQdvVNF5Gw8cffzzve9/7cs8996SnpycHHHBA7rnnnqxduzbz5s3LfvvtlyuvvDJLlizJbbfdlqGhoey555659NJL84IXvCBr167NySefnNNPPz1r1qzJiSeemFWrVmVkZCRr1qzJnDlzsnDhwqZaKh/7rr322qf92B133JENGzbkLW95S5Jk06ZNWb16dR5++OG8+93v3lmJAAAAAAAAsM1//dd/ZfPmzfna176WJHnkkUfyi1/8IkuWLMmXvvSlbZ935pln5n3ve1+S5Atf+EKWLVuWK664Ikny8MMP58UvfnHOPvvsbW8/9thj2z6/WZWPfc/k1a9+dW655ZZtb5977rk5+OCDc/rpp1dYBQAAAAAAwET2spe9LL/5zW9y0UUX5YgjjsgxxxzzlJ/3ne98JytXrsxjjz2W4eHh7T62yy675E1vetOYW7rHfAkAAAAAAAAwgey777752te+lqOOOiq33HJL5s2bl4GBge0+54EHHsjixYtz+eWX54Ybbsill16awcHBbR/fdddd09XVNeaWop/Z9//7yEc+UnUCAABtMjI8lMMWXd3SscODAzv+JADGhf6BoaxcuqDqDAAA2M6DDz6Y5z73uXnjG9+Yo446Kq973esyderUbNmyZdvnbNmyJb29vWk0GhkZGcmqVaue8TKnTp2a9evXP+uWWo19AACMH909vcW9YDwA5enb3J++9Ld0bKMxrc01AACUYnBoa0ceFDY4tDWTeyft8PPuvvvuXH755UmSkZGRLFy4MIccckgOOOCAzJ07Ny960Yty5ZVX5oQTTsicOXMyc+bMHH744bnjjjue9jLf+MY35stf/nLmzZuXOXPmZOHChU01G/sAAAAAAAColWYGuU5e7tFHH52jjz76Se//5Cc/ud3bH/jAB/KBD3xg29vnnHNOkmTWrFm59dZbt/vcfffdN9ddd92zTfaafQAAAAAAAFBXxj4AAAAAAACoKafxBAAAAACepH9gqOXXQhoc2trmGgDg6Rj7AAAAAIAn6dvcn770t3RsozGtzTUAwNNxGk8AAAAAAACoKc/sAwB2aGR4KIcturqlY4cHB9pcAwAAAAD8kbEPANih7p7ebNjQV3UGANBhY3l9LgAA2JlGhofS3dNbm8t9KmvXrs3JJ5+cW2+9dUyXY+wDAAAAknh9ronMmRwAqLNWH7A0OLS1AzXsLN09vblz6Rltv9xW/05UJWMfAAAAwK59d7YAACAASURBVATnTA4A1FmrD1jyYCXG6kc/+lGWLVuWRx99NElyzjnn5MADD8zJJ5+cU089Nd/+9rfz+OOP58Mf/nBe/epXJ0lWrFiRz372s2k0GjniiCPa0mHsA4DCeFQ1AAAAAJRt8+bNueCCC/KpT30qe++9dx566KG8+c1vzic/+ck8/PDDOfTQQ/Oe97wnX/nKV7Js2bKsWrUqv/jFL/KJT3wi1113Xfbaa69ceOGFbWkx9gFAYTyqGgAAAADKdtddd2Xt2rU588wzt72vq6srw8PD2W233XLssccmSQ499NAsWbIkSXLbbbflmGOOyV577ZUkOeWUU/L1r399zC3GPgAAAACK44wXTBTu6wD1NDo6moMOOigrVqzY7v1r167N5MmTt73d3d2d4eHhbcd0grEPAAAAYCfxj/rNc8YLJgr3dYB6euUrX5n7778/3//+9/Oa17wmSfLjH/84e+6559Mec+SRR+bqq6/Oxo0bM2PGjHzxi19sS4uxDwAAAGAn8Y/6AADjw3Of+9x8/OMfz2WXXZZLL700Q0ND2XfffXP++ec/7TEve9nLctZZZ+Wtb31r9tprrxxzzDFtaTH2AQBA4TwLBIC66h8YysqlC1o6dnBoa5trAGDn8Wdg543l/5V3dLndPb1Nfe4hhxyS5cuXP+n9t95667b/njVr1nZvL1iwIAsW/O99Y+HChWOofYKxDwAACudZIADUVd/m/vSlv6VjG41pba4BgJ3Hn4Gd1+wgV8rldlJ31QEAAAAAAABAazyzDwAAeNacWhQAAADKYOwDAACeNacWBQAAgDIY+wAAAAAA2E6rZ3JwFgeAnc/YBwAAsAP9A0NZuXRB1RkAADuNMzkA1IexDwAAYAf6NvenL/0tHdtoTGtzDROd18wEAAD+L2MfAAAA1IhnWgAAAP+XsQ8AAIBxbSynYR0c2trmGgAAgPYy9gEAADCuOQ0rAAAwnnVXHQAAAAAAAAC0xtgHAAAAAAAANWXsAwAAAAAAgJoy9gEAAAAAAEBNGfsAAAAAAACgpnqqDgAAABjPRoaHctiiq1s6dnhwoM01AAAAjDfGPgCgUv0DQ1m5dEFLxw4ObW1zDUD7dff0ZsOGvqozAAAAGKdqMfYtX748K1asSG9vbyZNmpTrrruu6iQAoE36NvenL/0tHdtoTGtzDQDAjnmwEgAAJSl+7PvGN76RG2+8MV/84hczderUbNiwoeokmuBURQAAAIxXHqwEAEBJih/7rrnmmvzDP/xDpk6dmiRpNBoVF9GMUk9VZIQEABi//F0PAACAiaj4se/Xv/51fvSjH+WjH/1oBgcHc+qpp+av//qvn/XlzJgxtW1N4+1ReJ26PiXeTt09vS0f2zN5lzQau7Sxpj1KvJ2TMrs0Na/Erk42TbTr26oSm5Jyu1pV4vXx81eGErtKbGpVp/+uV+JtVWJTUmaXpp2jxOs0kf5/OSmzq1NNg0NbWz4V61hNpPtViU1JmV0lNiVldmlqXoldmppXYleJTTyh8rFv/vz5Wbdu3VN+bM2aNdm6dWt+//vfZ+XKldm0aVPe+ta35oADDsjhhx/+rL7Oxo1bMjIymmTsd8hOPGOtyh+Sp7s+Jd5OSZldJX7/kjK7NG3P9695nbytSnsmcqMxrcjfVaXdTkm5t9VY+PkbuxKbkjK7Smwai079ThirErs62eR+1fzljkVpt1NS5nUqtcnPX3NK/P2ZlHk7l3hbua83r8TvX1Jml6bmldilqXnj/e+gRsP2q3zsu/baa5/x4zNnzszcuXPT3d2dGTNm5M/+7M/y4x//+FmPfQAAO8NEO41g/8BQy4+KHxza2uYaAAAAgImn8rFvR+bOnZvvfve7Ofzww/PYY4/lzjvvzHHHHVd1FgDAUyr1dWs7pW9zf/rS39KxHskHAO3lQTgAABNT8WPfO97xjpx//vmZM2dOkmTevHk56qijKq4CAAAASjfRnnHvQTg7x0S7XwEA5St+7HvOc56Tyy67rOoMAAAAoGYm2jPu2TncrwCA0hQ/9rFjHlEGAAAAwEThlLUAsD1j3zjgEWUAAAAATBROWQsA2+uuOgAAAAAAAABojWf2FcLpBwAAAKgzLzEBAADVMPYVwukHAOg0DywBADrJS0wAAEA1JuTY5x87AcYXv9eb44ElAAAAABOXf0Mbvybk2OcfOwHGF7/XAQAAAOCZ+Te08WtCjn0A7eJ1SQAAAAAAqJKxD2AMvC4JAAAAAABV6q46AAAAAAAAAGiNZ/YBsFM45SmMzVheRBsAAACA8cvYB8BO4ZSnMDZeRBsAAACAp2LsA2rDM8MAAAAAAGB7xj6gNjwzDAAAAAAAttdddQAAAAAAAADQGs/sg4o5NSUAtFf/wFBWLl3Q0rGDQ1vbXAMAAADQWcY+qJhTUwLQaRPtgSV9m/vTl/6Wjm00prW5BgAAAKCzjH0A49BE+4d94Jl5YEkZPOMQAAAA6ARjH8A45B/2AcrjGYcAAABAJxj7AAAAYCfzbF8AAKBdjH0AAACwk3m2LwAA0C7dVQcAAAAAAAAArTH2AQAAAAAAQE0Z+wAAAAAAAKCmvGYfAABMUP0DQ1m5dEFLxw4ObW1zDeOF+xUAAMDOZewDAIAJqm9zf/rS39Kxjca0NtcwXrhfAQAA7FzGPuApjQwP5bBFV7d07PDgQJtrAAAAAACAp2LsA55Sd09vNmzoqzoDAAAAAAB4Bt1VBwAAAAAAAACtMfYBAAAAAABATTmNJ0Ch+geGsnLpgpaOHRza2uYaAAAAAABKZOwDKFTf5v70pb+lYxuNaW2ugYnF2A4AAABAXRj7AAD+P8Z2AAAAAOrCa/YBAAAAAABATRX/zL577703H/zgB7N58+YMDg7mxBNPzNlnn111FmzH6d4AAAAAAIAqFD/2XXbZZTn++ONz+umn59FHH83cuXNz9NFH55BDDqk6DbZxujcAAAAAAKAKxY99XV1d6evrS5L09/enq6sr06dPr7hqYvBsNQAAAAAAgLJ1jY6OjlYd8UweeOCBnHXWWXn44YezefPmLFq0KAsWtDZAMX6ctmhFS8e1Ol7W1eDQ1kzunbTTj92REr9/JTaVym0FAAAAAFCOyp/ZN3/+/Kxbt+4pP7ZmzZp8/vOfz7x583LGGWfkoYceytve9rYcfPDBecUrXvGsvs7GjVsyMjL2XbPRmJYNG/rGfDntNNGaxnray4l0W41Fp7pK/P6V2DRWE+n7N1Yl/gxqal6JXZqaV2JXiU1JmV2amldiV4lNSZldmppXYpem5pXYVWJTUmaXpuaV2FViU1Jml6bmldilqXkldrWzyUtbtV/lY9+11177jB9fvnx5/uM//iNJsvfee+c1r3lNbr/99mc99gEAAAAAAMB40111wI7MmjUr3/3ud5MkW7ZsyZ133pmXvOQlFVcBAAAAAABA9Sp/Zt+OLF68OJdcckmuueaaDA8P58QTT8zRRx9ddRYAAAAAAABUrvix7+CDD86qVauqzgAAAAAAAIDiFH8aTwAAAAAAAOCpGfsAAAAAAACgpox9AAAAAAAAUFPGPgAAAAAAAKgpYx8AAAAAAADUVE/VAcDE0j8wlJVLF7R07ODQ1jbXPKHEJgAAAAAAaIaxD9ip+jb3py/9LR3baExrc80TSmwCAAAAAIBmOI0nAAAAAAAA1JSxDwAAAAAAAGrK2AcAAAAAAAA1ZewDAAAAAACAmjL2AQAAAAAAQE0Z+wAAAAAAAKCmeqoOgGerf2AoK5cuaOnYwaGtba4BAAAAAACojrGP2unb3J++9Ld0bKMxrc01AAAAAAAA1XEaTwAAAAAAAKgpYx8AAAAAAADUlLEPAAAAAAAAasrYBwAAAAAAADVl7AMAAAAAAICaMvYBAAAAAABATfVUHQBAffQPDGXl0gUtHTs4tLXNNQAAAAAAGPsAaFrf5v70pb+lYxuNaW2uAQAAAADAaTwBAAAAAACgpox9AAAAAAAAUFPGPgAAAAAAAKgpYx8AAAAAAADUlLEPAAAAAAAAasrYBwAAAAAAADVl7AMAAAAAAICaMvYBAAAAAABATRn7AAAAAAAAoKaMfQAAAAAAAFBTxj4AAAAAAACoqSLGvi9/+cs56aST8id/8if53Oc+t93HHn/88fzjP/5jjjvuuJxwwglZvXp1RZUAAAAAAABQlp6qA5Jk9uzZueKKK/KpT33qSR/79Kc/nSlTpuTmm2/OfffdlwULFuQb3/hGpkyZUkEpAAAAAAAAlKOIZ/a99KUvzYEHHpju7ifnfP3rX8+pp56aJNl///1z8MEH5zvf+c7OTgQAAAAAAIDidI2Ojo5WHfFH5557bg4++OCcfvrp2973yle+Mv/5n/+Z6dOnJ0kuvPDC7Lfffvmbv/mbqjIBAAAAAACgCDvlNJ7z58/PunXrnvJja9asyaRJk3ZGBgAAAAAAAIwrO2Xsu/baa1s+dubMmXnggQe2PbPv97//fY488sh2pQEAAAAAAEBtFfGafc/khBNOyOc///kkyX333Zef/OQned3rXldxFQAAAAAAAFSviNfsu+GGG7J06dJs3rw5vb292XXXXXPNNdfkwAMPzGOPPZZzzz03P//5z9Pd3Z1//ud/zhvf+MaqkwEAAAAAAKByRYx9AAAAAAAAwLNX/Gk8AQAAAAAAgKdm7AMAAAAAAICaMvYBAAAAAABATRn7/h977x0V5bE//r+WYgOuRoMlirFFiS1Y0dgVOwhi0Fhilyi22JWoGFERjWKHYCJ2jQXBrjcaNRgvapoaJUYEsaIIKCBSn98f+9u9rKDxfr6fz854M69z9rg8j+fs68zMzjM775n3KBQKhUKhUCgUCoVCoVAoFAqFQqFQKBRvKCrYp1AoFAqFQqFQKBQKhUKhUCgUCoVCoVC8oahgn0KhUCgUCoVCoVAoFAqFQqFQKBQKhULxhqKCfQqFQnoyMjL4/fffSU9PF60iJcnJyWRmZgLw4MEDjh07RmxsrGArheLvifruvR63b9/m+PHjxMXFiVZRKBQKhUKhUCgUiv8anj59SkZGhmgNI1euXOG7777ju+++48qVK6J13jiePHkiWkHxBqGCfX/BxYsX8ff3x8fHBx8fH/z9/bl48aJorULMmTNHyOempKTw+eefM3z4cLZt22Zyb/z48UKcDFy/fp0///wTgPj4eDZu3MiPP/4o1OlF5s6dK1rBWEYAOTk5rFy5kiFDhrB48WJjAMnczJ07l+TkZAB++uknOnfuzPTp0+ncuTNRUVFCnL7//ntycnKEfPar2LlzJ506daJr166cOHGCfv36sWfPHoYOHcru3buFOOXk5BAaGkqvXr1o2rQpLVq0YODAgZw8eVKIT0Hu3bvHhg0bWLhwIQsXLmTDhg3cuXNHtFaRjBw5Usjnqn799cnMzCz0GjVqFM+fPxfWf549e9b4Pi0tjWnTpuHi4sL48eNJSkoS4gQwevRoY79+4sQJ+vfvb+yrIiMjhTjJ2q/n5+dz9OhRfvnlFwCOHz+Ov78/O3fuJD8/X5hXYmIiGzduZOHChQQGBrJnzx6ysrKE+cCb1aeD6ffT3Pz0008cOHCAR48emVzft2+fICN9/Y0fP56JEyfy6NEjvvjiCxo3bkz//v2lqsdPP/1UtIIJ586dY+3atcLHVT/88INx8unp06f4+vri5ubGtGnTjP29udm6dauwz34Vly9f5t69ewBcunSJDRs28MMPPwi20rssWLAAHx8fJkyYwMqVKwv1EebmTZl/MSBqHiY7O5vg4GDmzJnDqVOnTO75+/sLcQL5xuovQ7Z+/cmTJ1y8eJHHjx8L9di7d6+xD33w4AFDhgyhcePGDBgwgISEBCFOMo7XY2Nj8fb2Zu7cuTx58oTRo0fTqFEj+vXrJ3Th59OnT/Hz86Nx48Y4OzvTtGlT2rdvz5YtW4Q5Xbp0iS5dujBr1ix27drFrl27mDVrFl26dOG3334T5vUyRNbflStX6Ny5Mw0bNmTChAkm45mhQ4cK8wL9vIJhzPfkyRP++c9/cvPmTaFOipej0zRNEy0hK+vWrePo0aN4eHhQqVIlAO7fv09ERARdu3Zl7Nixgg3/Tfv27QsN8szBhAkTqFKlCk5OTuzYsQMbGxtWrFiBlZUVHh4eREREmN0JYMuWLYSFhZGbm8uIESOIjIykQYMGREdH88knnzBw4ECzOy1ZsqTQtd27d+Pl5QXA9OnTza0EQO/evY0TPcuXL+fGjRt4eXlx7NgxrK2thfxY6NWrF/v37wdg8ODBTJ06lYYNGxIXF8eUKVMIDw83u9P7779P6dKlcXNzo0+fPjg6OprdoSjc3NzYsGEDGRkZeHp6snfvXqpXr86jR48YPnw4Bw4cMLvTrFmzsLKyom3bthw5coRq1arx3nvvsX79evr06SPk+wf679uaNWtwcXEx6dNPnDjB2LFjjd9Fc/KqgFC3bt04ffq0GW30qH799XF0dESn01HUUEqn03Ht2jWzOxXs0+fPn09+fj4DBgzg0KFD3Lp1ixUrVpjdCcDd3d0Y1Pv4449ZsmQJVatW5fHjxwwbNszY55sTWfv1+fPnc+XKFXJzc2ndujXnz5+nU6dOnDt3jho1ajB79myzO+3fv5+goCAcHR355ZdfaNmyJRkZGfz555+EhIRQp04dszvJ2Kf/FaLG62FhYezYsYMaNWpw6dIl/Pz86Nq1K2DaZ5ibUaNG0aZNG9LT0zly5Aiurq706dOHI0eOcO7cOdatW2d2p4kTJxa6dubMGdq2bQvAypUrza1Ev379+PbbbwGIiIhg7dq1uLi4EBUVRe/evRk+fLjZnQBcXV3Zv38/FhYWzJ49m1KlSuHq6soPP/zA1atXWbt2rdmdGjZsiKWlJS1btqRPnz60b98eS0tLs3sUZM2aNYSHh6PT6RgxYgSbN2+mRYsW/PTTT/To0YMxY8YI8dqwYQORkZE0a9aMqKgonJ2dsbCw4LvvvmPZsmU0b97c7E5v0vyLAVH9uq+vL5mZmTRs2JC9e/fSsmVLPv/8c0Bcvy7jWB3k7Nf9/f2NgeJff/0VHx8fKlasyL1791i6dClt2rQxuxPo+/WDBw8C8Nlnn+Hk5ESvXr04c+YMERERbNy40exOMo7XBw4cSLdu3UhLS+PIkSN4eHjg4eHBqVOniIiIEBZcGzNmDA0bNqRdu3YcOHCAt956ixYtWrB69WoaNGjAhAkTzO7k4eHBnDlzaNKkicl1w8IOUYs+X4aoPh1gwIABeHt74+TkxNatWzl06BAbNmygUqVKQudhDh8+zJw5c7CwsGD+/PkEBwdTvnx5YmJimD17Nt26dRPipXgFmuKldO7cWXv+/Hmh65mZmZqLi4vZfVq0aFHky9nZWatbt67ZfTRN03r16mV8n5+fr82bN08bPny49vz5c83d3V2Ik6Zpmpubm5aRkaE9evRIa9iwoXb//n1N0zTt8ePHwrw++OADbcqUKdrq1auNr6ZNmxrfi6JgeXh4eGjp6emapmlaTk6O1rNnTyFOXbp0Mb739PQ0uefq6mpuHU3T9OV07do1zd/fX3N2dtZ69+6tbd26VXvy5IkQHwMeHh7G9+3btze5J6qtd+/e3fg+JydH+/jjjzVN07TU1FSta9euQpw0Td+uHj9+XOj648ePtc6dOwsw0rQ6depojo6OWp06dYwvw9+Ojo5CnFS//vrMnDlT8/X11dLS0ozXOnToIMTFQMGy6NWrl5adnW38W1T/qWn6719eXp6maZrm5eVlck/166Z0795dy83N1dLT07VGjRoZ21dWVpaw53KPHj2M/WdCQoI2evRoTdM07ccff9QGDBggxEnGPl3TNC0wMLDI1+LFi7XGjRsLcXJ1dTW2oxs3bmhdu3bV9uzZo2mauLGCppk+b1q1amVyz83Nzdw6mqZp2ocffqjNmjVLCw8P18LDw7W9e/dqzs7Oxr9FULCO+vbtq92+fVvTNE1LSUkR2q/36NHD+L7geFTTTOvWnLi7u2uPHz/WNmzYoLm6umqtWrXSAgMDtRs3bgjx0TR9OT179kxLSkrSnJyctIcPH2qapmlpaWnC+nRN0z9rnj17pmmavt8cNmyYpmmadu3aNa13795CnGSbfzEg4zxMwe9+ZmamNmbMGG3WrFlafn6+sH5dxrG6psnZrxfsM4cPH66dPXtW0zRNu3r1qvbRRx8JcdI0zeS3+ov9uqg6lHG8XvAZ17ZtW5N7Itv6i2OCvn37apqmac+fPzeZYzMnr/pcUU5bt2596at58+ZCnDStcNs5ePCg1rlzZy0hIaHQ99GceHh4aImJidrNmzc1Jycn7fr165qm6X8Tvjhfq5ADlcbzFWiahk6nK3TdwkJMsWmaRlhYGHv27Cn0Klu2rBCn7Oxs43udToefnx+1a9fG29tbaGonCwsLSpUqxdtvv42DgwMVK1YEoGzZskXWqTkIDw8nMTERGxsbxo4dy7hx47Czs2PcuHGMGzdOiBPo25Uh5ZylpSU2NjYAWFlZYWVlJcSpZcuWxjSizs7OHD58GNCnvypTpowQJ51Oh6OjI7Nnz+aHH35g5MiRfP/997Rr144pU6YIcQKwtrbm9OnTHDx4EJ1Ox7FjxwD9SilRfZWlpaXx+5+enm58X7p0aaytrYU4gT41XlF95VtvvVXkzixzYG9vz9mzZ4mJiTG+rl27RkxMDOXLlxfipPr11ycgIAAXFxeGDh1q3IUpysVAdnY2sbGx3LhxA51OZ/KdE9UnAHTv3p0pU6Zw584dXFxcCA0N5cGDB+zatYvKlSsLcZK1X7eyssLS0pJSpUpRsmRJbG1tAShWrJjQft3Qfzo4OHD//n1A/7xOSUkR4iRjnw76nQ3FixenVKlSJi8bGxuh/YOhHdWsWZPNmzcTFhbGzp07hToV/Oy6deu+9J45OXDggPGs6G7duuHp6UmpUqXo3bs3vXv3FuJUsCxycnKoUqUKAGXKlBE2Vgf9d+3SpUuAfjxjSD+XkZEhLOWwTqejbNmyDBs2jAMHDrBu3ToyMjL4+OOP+fjjj4U4WVtbU7JkScqVK4ednR329vaA/jspsv4sLS0pWbIkAP/4xz+M9efo6GgyFjQnss2/GJBxHiYvL8/4vkSJEqxevZrMzEymTZsm7Psn41gd5OzXC5KUlMSHH34I6Hexifr+AVStWtWYIrpq1arEx8cDCE3vK+N4PS8vj6ysLJKTk3n69Kkx3WJmZqbQ38s6nY7U1FQA7t69a+wLihcvLux5U6VKFUJCQoxeAKmpqQQHB/POO+8IcVq0aBGXL1/mypUrhV4iv39ZWVkmfXvPnj2ZNm0aQ4cOFZ6ivHz58lSvXp2KFSvy3nvvAfrfhAV9FfIgbnT5BuDh4YGXlxceHh7GTujevXtERETg7u5udp/69euTkpJS5Lb1ChUqmN0H9F/uCxcu0KxZM+O1GTNmEBQURGhoqBAnwGSAO3nyZJN7ovJ916hRg02bNhEaGsrgwYOZN2+e8ElhgD/++INGjRoZf1wlJiZSoUIFsrKyhP1Q8PX1ZcmSJbRt25YyZcqwYcMGpk+fjrOzM4sWLRLiVHDi0Nramh49etCjRw8ePHggbDs9wOeff46fnx8WFhasW7eOr7/+mhkzZlC8eHGCgoKEOLm6utK3b1+aNm3K2bNnjSlbRJ4XBtC6dWtGjhxJ3759Tfr0Xbt20apVKyFOzs7O/Pnnnzg7Oxe617BhQwFGql//T+nQoQNOTk74+/tz5MgR4QPe58+f4+3tbeyzDH16enq60MmyiRMnsnHjRgYMGEBKSgo5OTl89dVX9OzZk4CAACFOsvbrlSpVYsmSJWRkZFCzZk0WLVqEm5sbP/zwA2+//bYQpypVqhAcHEybNm04dOiQ8UdeXl6esDYvY58OULt2bbp27VrkeF3UWbrW1tY8evTIGGQoX748mzZtYvjw4dy6dUuIE+gnqNPT07G1tTV5vqSmpgpLvVi2bFlWrlzJ/v37GTRoENOmTRM+Xo+Li+Ojjz5C0zQSEhKMZQZin3+ff/4548ePp2nTptjb29O3b19atGjBpUuXhJ07/GKgv2HDhjRs2JBZs2bxz3/+U4iTnZ0d27ZtIz093fi7xsPDgzNnzlCiRAkhTqAPsM+ZM4fWrVtz7NgxY4q1rKwsYe1KtvkXAzLOw7z99tvExMQYnSwtLVm2bBkzZswwnplnbmQdq8vYrycmJrJkyRI0TePJkyfk5eUZn3siz2f28/Nj3LhxhIWFUbp0aby8vKhXrx73798Xdj6ljON1Nzc3unfvTm5uLuPHj2f8+PHUqVOHn376iU6dOglxAhgyZAi9evWibt26XLlyBV9fX0A/DyMqsLZkyRKWLVtGhw4dTK5369aNpUuXCnGqWbMmo0aNombNmoXuiTxjtGXLlkRFRdGuXTvjtc6dO2NlZWWsSxEU/K03ZMiQl95TyIM6s+8vuHjxIocPHzYeqP3OO+/QrVs3ITnss7OzsbS0FH7uQEFSU1PR6XSULl260L0bN25Qq1YtAVb6XXRdu3Y17lIzcPPmTb799ltmzZolxMtATEwMc+fO5fbt25w7d06oy8t4+vQpN2/exMnJSZjDs2fPSEhIIC8vj3feeYe33npLmMuMGTMIDAwU9vn/CSkpKZQuXVroxL5ht1q9evVo0aKFMI+C5Ofns3//fo4cOVKoT3d3dxe+algWVL/+P+fo0aNER0fj5+cnWqUQmZmZJCUl4eDgIFqF9PR0cnNzhe3UNiBrv56cnExISAg6nQ4fHx+OHj3K1q1bqVKlCr6+vkLqMCkpiYCAAP744w/q16/P9OnTKVu2LKmpqVy6dMl47o05kbVPP3v2LFWrVi2ynn766adCZ5aYgxMnTmBvb19oEUlKSgrr168Xdm70y3bxJCcnk5SURO3atQVY/ZvExETmzp3Lr7/+SnR0tDCP8+fPm/xd4E4ARgAAIABJREFUr149bGxsePToEcePHxd2Dhbony0HDx7kxo0bxvF69+7djeetmZtly5YJ3ZldFPHx8SxduhQLCwtmzZrFt99+y+bNm3FwcCAgIIB69eoJ8Xr27BlfffWVcbzu7e1tDMDfuXNH2LlYFy5cKLJfFzH/YkDGeZj4+Hisra0LZUfQNI0zZ86YTBabizdhrC5Lv75mzRqTv/v370+5cuVITEwkKCiIxYsXCzLT8+OPP5r0623btjXuBDY3so7XY2JiAP1u6Hv37nH06FGqVKlCly5dhHrFxsby559/4ujoSLVq1YS6vIhhd5/o34D79++nbt26Rc5rHDp0iJ49ewqwkpeQkBAGDRpkXGhmIDY2ltWrV7NixQpBZoqXoYJ9CoUgcnNzefTokbAfwwqFQqFQyEBGRkahiSGFQqFQKBQKhUKhUPx3sW7dOnx8fERrvBHMnTuX+fPni9ZQvGGobQwKhSCsrKxUoE+hkISzZ8+KViiEqBRYr8LNzU20QpGo+ns9ZK0/GVdPytimQE4vkU737t3j559/LnS+huhyktFr//79hISEGFeiG/jqq68EGel3Fn7++ecMHz6cbdu2mdwbP368ICv9DswDBw4UOqNo3759gozkrD+Q00vG+svOziYxMbHQdVHpFg3IWFbff/+98RzkixcvsmDBAmEpkA2kpaURGBhoTLH99ddf06tXL6ZOnWpyDpU5uXfvHuPHj2fixIk8evSIL774gsaNG9O/f3/u3LkjxAng+vXrxnYdHx/Pxo0bhabFMyBju5KxrGRs6yBfWclaTjL2C9u2bSv02rx5s/G9LMydO1e0AkuWLCn0OnLkiPG9TMhQXoqXo3b2KRQKheJvT/v27Tl16pTZPzczM/Ol97p162b8UWpObty48dJ7Q4cOJSoqyow2r4eqv38ja/29qixmzpwpXUprUW3qr5DRS5TT/v37WbRoEfb29qSnp7N8+XIaNWoEQO/evYVNVsvotWTJEn799Vfq1q3LsWPHGDFiBEOHDhXqBDBhwgSqVKmCk5MTO3bswMbGhhUrVmBlZYWHh4eQs3jCwsLYsWMHNWrU4NKlS/j5+dG1a1dA1d+LLF26lF9++UUqLxnrLyoqikmTJqFpGlWrViUoKIh3331XqBPIWVYrVqzg7Nmz5Obm0qJFC65cuUKbNm344YcfaN68ubBFAJ999hkVKlTg2bNn3Lp1i5o1a+Lh4cGxY8dISkoSMgk7atQo2rRpQ3p6OkeOHMHV1ZU+ffpw5MgRzp07x7p168zutGXLFsLCwsjNzWXEiBFERkbSoEEDoqOj+eSTT4SlHJaxXclaVjK2dRnLSsZyAjn7hbp169K2bVuTI3mOHj1Kt27dAISc3V5U/ezevRsvLy8AYentnZyccHFxMUm/umnTJuM5eePGjRPiJWt5KV6OlWgBhUKhUPzv8uzZM0qVKiVaQzpeNujWNI20tDQz2+hp1KgROp3O5OBxw9+iDo53dXWlcuXKFLUWSORKRVV/r4es9Td69GiaNWtWpFdGRoYAIznbFMjpJaPTN998Q2RkJBUqVCA6OprJkyfj7+9P69ati2xnf2evM2fOsG/fPqytrRkzZgw+Pj6kp6czbtw4oWV169YtVq1aBUDnzp2ZP38+n376qZDJKAPh4eGEh4dja2tLbGwsY8eOJT09nT59+qj6e4HTp09L5yVj/QUFBbFlyxYcHR3Zt28fw4YNY926dTg6OgqtPxnL6sSJE0RERJCZmUnr1q05deoUZcqUYdCgQfTr109YsC82NpYVK1aQl5fHhx9+SFhYGJaWljRs2JBevXoJcXr48CGDBw8GYPv27Xz66acAfPLJJ8J2rO3evZuDBw/y7NkzOnXqxLFjx6hYsSLJyckMHz5cWABLxnYla1nJ2NZlLCsZywnk7BfCwsJYtmwZnp6exvMMo6OjhQT5DGzfvr1QUA0QPocWHh6On58f9erVY+jQoeh0OsLDw4UF+QzIWl6Kl6OCff8hBw8exNXVVbSGkcWLFzNz5kzRGiYMGjSIrVu3itYoxPnz54Ue7F0UMjoBTJ48meXLl4vWMEHGdiVr/Q0cOFBo+p2ikKGv2rJlCyNHjsTS0rLQPVGBGXt7eyIjIylbtmyhe+3atRNgBJUrV2b79u1UqFCh0D1RTqDq73WRtf7effddFi5ciIODQ6F7orxkbFMgp5eMTpqmGdu5s7Mz69evx9vbmzlz5gitP1m9rK2tAShXrhzffPMNY8aMISsrS6hTwTSnOp0OPz8/AgMD8fb2JisrS5iXra0tADVr1mTz5s0MHz6cnJwcVX9viJds9ZeXl4ejoyOg3zFXuXJlxowZw8qVK4XXn2xlZWVlhaWlJba2tlStWpUyZcoA+gnFop4/5vQCsLS0pFKlSkYXnU6HhYWYk3EK1lHdunVfes+cWFhYUKpUKUqVKoWDgwMVK1YEoGzZskLbuoztSuayArnauoxlJWM5GT7fgCz9grOzM2FhYSxevJjDhw8zd+5c4c8+WYNqNWrUYNOmTYSGhjJ48GDmzZsnvKxA3vJSvBwV7PsP+eabb6QK9kVHR4tWKISoFfp/RUBAgHQBEBmdAOLi4kQrFELGdiVr/cmYnVmGvqp27dp07drVOOFSEFEr3Zydnfnzzz9xdnYudK9hw4YCjKBLly7cvXu3yGBR586dBRjpUfX3eshaf3379uXJkydFBvsMK1DNjYxtCuT0ktEJ9Gem2NnZAVCrVi02bNjAyJEjefLkiTAnGb1sbW1JSEigatWqxr/Xr1/Pp59+yvXr14U4ATg4OHDhwgWaNWtmvDZjxgyCgoIIDQ0V4mRtbc2jR4+wt7cHoHz58mzatInhw4dz69YtIU6y1p+MXjLWX25uLllZWRQvXhyA5s2bs3z5ciZOnCg0qC1jWeXn5xuzIyxatMh4XdM0cnNzhTiBfkLfUIcF0ws/e/ZM2ERsiRIlSE9Px9bW1qS/TE1NFRbAys/PN76fPHmyyb2cnBxz6xiRsV3JWlYytnUZy0rGcgI5+wUAGxsb/P39OX36NMOGDeP58+fCXEDeoBrog9ujR4+mffv2zJo1i2fPnolWkrq8FC9BU/xHuLu7i1YwQTYfTdM0Ly8v0QpFImNZyeikaXJ6ydiuZCwnTdO0sWPHilYohAxlFRUVpSUkJBR57+LFi2a2UfynqPpT/G8ja5uS0UtGp127dmnnz58vdD0hIUGbMGGCACM9Mnr9/PPP2p9//lnoelZWlrZ161YBRnpSUlK01NTUIu8V5WsOvvvuO+23334rdD05OVkLDAwUYCRv/cnoJWP9rV27VouKiip0/bffftP69OkjwEiPjGV15swZ7dmzZ4Wux8XFaV9//bUAIz0PHz7UcnNzC12/f/++9uOPPwow0rT8/Pwirz9+/Fj7448/zGyjZ+/evVp6enqh67GxsdqiRYsEGOmRsV3JWlYytnUZy0rGctI0OfuFF3n69KlUv92vXbumeXl5aS1atBCtUoicnBzt3r17ojVMkLm8FP9Gp2kSbgGRmPDwcDw9PUVrGLl+/Tq1a9cWrfFGEBISwujRo0VrmCCjE+hX/hjSWyhejqz1JyOqr1IoFAqFQqFQKBQKhUKhUMhCbm4ujx49olKlSqJV3ghUecmPCvYpFAqFQqFQKBQKhUKhUCgUCoVCoVAoFG8o4k4OVSgUCoVCoVAoFAqFQqFQKBQKhUKhUCgU/0+oYJ9CoVAoFAqFwuzIcOC4QqFQKBQKhUKhUCjMQ3x8vGiFN4bz58+LVlC8gahg3xvIrVu3RCsUIjQ0lJSUFNEaiv8H1q9f/1rXRHHw4EHRCtJz7tw5tm7dCkBSUhJxcXGCjeRHxsHT5MmTRSsUYtCgQaIVikTV3+sha/0NHDhQtEIhZGxTIKeXjE6yjhVk9Fq8eLFohSKRsb/asGGDaIVCyFp/MnrJWH8y9p8gZ1nJ2H+CnG1dxjGorG1dxnYla1nJ2NZlLCsZywnk7BcmTZokWqEQMrYpgICAANEKRSJreSn0qGDfK1iyZMlfvtatW2d2Lx8fHzp06MCsWbOIjIwkMTHR7A4v8vDhQ3r27Mn06dP57bffROsYOXz4MOnp6QCsXLmSESNGcOXKFeVUBIcPH36ta6L45ptvRCsAkJmZSVBQEFOmTAEgNjaW7777TrCVPuC+Zs0aNm/eDOgPzfX19RXicuTIEZo1a8aHH34oVRsqChkHTzIGaTMyMkQrFImqv9dD1vqT8dhoGdsUyOklo5MsY4UXkdErOjpatEKRyNhfHThwQLRCIWStPxm9ZKw/GftPkLOsZOw/Qc62LuMYVNa2LmO7krWsZGzrMpaVjOUEcvYL6jfg6yNjWYG85aXQo4J9r2D//v2UKlXqla/IyEizex06dIhdu3bRokUL/vWvf+Hl5UWPHj3M7lGQ2bNnc+LECRo1asScOXPw9PRk7969ZGVlCfUKDg7G1taWS5cuERUVhYeHBwsWLFBOBTh79iyBgYE8fPjQJJA9d+5cYU5FIctDbt68eeTl5RETEwNAxYoVWbNmjWAr/erEjRs3UqpUKUDvZQgqm5uVK1eyY8cOQkNDCQoKYt26dVy/fp309HRu374txOllyNKuCiKjk7W1tWiFIpGxrGR0krX+qlSpIlqhEDLWH8jppZxeHxm9ZHQCOfsrGctKRieQ00s5vT4yesnoBHJ6KafXR0YvGZ1ATi/l9PrI6NW4cWPRCoWQsZwAunXrJlqhSGQtL4UeK9ECMtOqVSvGjRv3yv8jYuJa0zTu37/PvXv3uHv3LmXKlKFJkyZm93iRkiVL0q9fP8qVK8eiRYsIDQ1l5cqVzJw5U1gw0spK38TPnj2Ll5cXbm5uwtOTyOZkbW2NjY0NOp3OGCgCKF++PN7e3sK8XmTw4MGiFQC4fv06gYGBREVFAWBjY0N+fr5gKyhRokShCTKdTifEpVixYtSqVQuAbdu2sXDhQiIjI5k5cyY7d+7kq6++EuJVFDIOnjZu3ChaoRC7du0SrVAkqv5eD1nrT4aFEi8iY5sCOb1kdJJlrPAiMnotWbJEtEKRyNhfTZ06VbRCIWStPxm9ZKw/GftPkLOsZOw/Qc62LuMYVNa2LmO7krWsZGzrMpaVjOUEcvYLsm0sADnbFMDo0aNFKxSJrOWl0KPTVDj2jaNp06a899579OvXj5YtW1KhQgXRSiQlJbFz507Cw8Np0KABgwYNolmzZty+fZtPPvmEU6dOCfHy9PRk6NChhIaGEhwcjIODA66urkJztMvolJeXx44dO6Q8J0U2+vbty65du/Dw8CAiIoKsrCw++ugj4WlvJkyYwODBg1m4cCF79+4lJCSE2NhYli1bZnaXUaNGMXHiROrXr2/2z1YoFAqFQqFQKBQKhUKhUCgUir8blvPmzZsnWuJNISEhgX379pGUlESNGjWEeeh0OhITE/nhhx+4ffs2WVlZlCtXjpIlSwpz6tKlC7Vr12bevHl4eXlRuXJlAEqXLs3Tp09xdnYW4lW7dm127dpFly5daNu2LfHx8Tx+/Jh27doJ8ZHVycLCgsDAQPr06SPM4U0hLi6Oy5cvc/XqVWrVqkVAQADNmjWjZcuWQr2aNm1KQEAAly9fJiQkBAB/f3+T3ZrmonXr1pQsWRIbGxuzf7ZCoVAoFAqFQqFQKBQKhUKhUPzdUDv7XsHQoUOZOXMmjo6OPHjwADc3N5ycnLhz5w7u7u7Ct9Pm5ORw8OBBVq9ezYMHD7h69aowl6ysLIoXLy7s84siLy+PtWvXMmHCBNEqbwRr1qyhVq1aajv2X5CTk8PXX3/NyZMn0TSNjh074u3tbUzPKprMzEzy8/NVoE2hUCgUCoVCoVAoFAqFQqFQKP4mWIgWkJmHDx/i6OgIwP79+2nZsiXr16/n22+/5dChQ8K8jh07xrx583B1dWXt2rV8+OGHLF26VJiPgd27d7N8+XKWLFlifInE0tKSCxcuCHUoivj4ePr370/Hjh0B+P3331m9erVgK9i6dSufffYZTk5OtGzZkhYtWgjfrSYj1tbWjBkzht27d7Nnzx58fHykCPQNHjyY/fv3o9PpVKDvDWT9+vWvdU0UItMMv2k8fvyYX3/9VbQGoaGhpKSkiNYoxI0bNwq9EhMTRWsp3mBiY2Nf65rCFFnLSD1vFAqFbBiOupBhzkXx30N+fj6nT58WraH4H/Am9AlPnz7l6dOnojVMSEhIICoqitOnTxtfCoXifx/xM9QSU3Cn2s8//4yLiwsA//jHP7C0tBSlxfHjx2nRogUjR46kSpUqwjwKMnHiRHJycmjYsCHFihUTrWOkffv2fPPNN3h4eJikMxSZ8nTevHmMGTPGeJba+++/z/Tp0xk/frwwJ4C9e/cK/fw3haKC2HZ2dsYgqSiGDRvGvn37CAgIoFOnTnh6etK4cWNhPjKybdu2v/w/pUqVonfv3mawMeXw4cOMGjXqL6+J4ptvvsHV1VW0BqDfvRoSEsKdO3dYtmwZsbGxxMXFGZ/RIhgwYABfffUVmqbh4eHBP/7xD9q2bcuMGTOEOT18+JCePXvSunVrBg4cyAcffCDMpSDe3t7cv38fOzs7ANLS0ihXrhzFihVj+fLlODk5CTbU8/vvv1OvXj3RGoWQ0Uu009SpU9m3b99fXjM3YWFhfPTRR9jZ2TFt2jQuX77M7Nmzad26tdldMjMzC10bNWoUhw8fRtM0oePiF5HlebN48WLGjh1LyZIlGTx4MFevXuWLL77A3d3d7C59+vRBp9O99P6ePXvMaFOYmzdvEhwczO3bt8nNzTVeF+n1+PFjtm7dSkJCgonTypUrze7yV+PPgQMHmsmkaCZMmFCofRl+23h6emJhYf714Y8fPyYgIID79++zbds2YmJi+OWXX+jfv7/ZXQw+KSkpREVFMX78eF5MjiWiD01JSeHLL7/k/v37dOrUyaQdjR8/XorFxACffvopX331lWiNIvnuu++oWLGisHPmLSwsCA4OFnqcy8tIS0tj/fr1XLt2jaysLOP1zZs3m93lrwJCIspPxj4BIDk5mS+//JIjR44AoGkaOp2O7t27M3XqVMqWLSvEC/TzaBEREVSvXt34XNHpdELb/+HDh2nbti22trasXLmSS5cuMWnSJGF9QlHPY8O4ZenSpUybNk2EltTzaIqiUcG+V2Btbc2ff/5JuXLluHDhArNnzzbeK/jAMzeGINGzZ8949uyZkDO5XuTWrVvGB4pMGFbaLF26FJ1OZ3zYXbt2TZhTWloabdu2Zfny5YB+kGdtbS3Mx0DlypVJT0/n1q1b0k0kysTjx4+5ePGiMbBw4sQJmjRpwpEjR+jevTtjxowR4tWhQwc6dOhASkoKhw4dYuHChWRkZHD06FEhPjKyatUq447al3HhwgWzDlLOnj1LVFQUDx8+NAkkp6enm83hdZAp4/e8efOwt7cnJiYGgIoVKzJlyhShwb5nz55hZ2dHZGQkbm5uTJ06FXd3d6HBvtmzZzNlyhQiIiKYM2cOVlZWDBw4EFdXV6Fptzt16oSzs7Oxvr777jsuXbpEq1atWLhwIbt37xbmVpCVK1cSGhoqWqMQMnqJckpOTiY5OZmsrCxiY2ON/VRaWhrPnj0zu8+LhIeHM2zYMP71r3+RnJzMokWLWLBggZBgX6NGjYzj4II4OTkJHxe/iCzPmx9//JGZM2dy6tQpKlSoQFBQEN7e3kKCfYZnyalTp7h58yYfffQRoG9jdevWNbvPi0yePJlu3brh6ekpdEFsQXx8fKhbty4tW7YU7nTlyhVAH5w5f/68cXHguXPnjAtyRGJvb8+VK1fo2bMnoJ/8tLe358iRI8TExJjMgZiL2bNn07ZtW7Zv3w5AjRo1mDZtmrBgX5cuXWjfvj3Z2dnGRUmi5xb8/PyoUqUK7dq1Y8eOHZw7d44VK1ZgZWXF7du3ze4D+gXgL3L+/HnjdRHB9lfxz3/+k99//50KFSrwzTffCHGoX78+ly5domHDhkI+/2X4+vpSs2ZN4uPjmThxInv37hU2R/T111+/9J6oYJGMfQLAtGnTaNq0KSdPnuStt94C9OPlnTt3Mm3aNGHtHPS/+U6cOCHVArPg4GB69OjBpUuXiIqKYvDgwSxYsICdO3cK8enQocNL7zVp0sSMJqbIOI+meDUq2PcKJk+ezKBBg8jMzMTLy8u4i+7s2bNUr15dmNft27eZMmUK165dQ6fTUbduXZYuXYqDg4MwJwcHB9LT07G1tRXmUBSGyWCZsLS0JCcnx7hiIzExUciKyRc5ffo0c+fOxdLSkpMnT3L58mXWrl1LSEiIaDWpePjwIeHh4ZQuXRrQTybMmDGD7du307dvX2HBPgOGdqVpmjQTZrJQt25dAgICXvl/hg0bZiYbPdbW1tjY2KDT6UwWbpQvXx5vb2+zuryKwYMHi1Ywcv36dQIDA4mKigLAxsaG/Px8oU7Z2dkAREdH06NHDywsLIRPLoJ+VWm/fv0oV64cixYtIjQ0lJUrVzJz5kx69OghxOn8+fN8/vnnxr9dXFxYv349kydP5vnz50KcikK2gJoBGb1EOR04cIBNmzbx8OFDk13QdnZ2jBw5UohTQQx9QHR0NG5ubjRu3FjYc7l3795YWFgwa9Ys41i9Y8eOnDx5UojPq5DpeQP6yYvOnTtToUKFV+6u+7+kefPmAKxevZrNmzcbPTp06MDQoUMZN26cEC8D+fn5ws+yf5HMzEz8/PxEawAYx55jx44lMjLS+Jv99u3bxsWfIomJiWHLli3G7Dz9+vVjzJgxhISE4OHhIcQpMTGR/v378+233wJQrFgxob+XJ02axKRJkxg4cOBr7XAwB7du3WLVqlUAdO7cmfnz5/Ppp5+ybt06YU4XL16kXbt2NGvWDND/Ho2OjqZ9+/bCnF5FYGAgAE+ePBHmcPHiRXbs2MG7775r8ltQ9I7tW7dusXr1ak6cOIGrqytdunQR9tt0y5YtQj73VcjYJwDcvXu3UECvbNmy+Pj40LVrV0FWeipVqiTFJoeCGI7jOXv2LF5eXri5ubFhwwZhPq8Klv1VsO3/Ehnn0RSvRgX7XkGLFi348ccfycjI4B//+IfxeqNGjahVq5Ywr7lz59K3b1/69OkD6Fd1zp07l7CwMGFOdnZ29OnThzZt2pik8Zw+fbowJwNxcXHExsbi4uJCRkYGOTk5lClTRpjPgAEDGDduHCkpKaxevZqIiAgmTZokzMfAqlWr2LNnj3HCrEGDBiQkJAi2ko/ExERjoA+gdOnS3L17F1tbW6EpbE+ePMm+ffu4ePEinTp1wtfXl6ZNmwrzkZHX6SPN3Y82b96cJk2aUKZMGWPufxnx9PQUrWDkxR8JWVlZwgPbzZs3p2vXrmiaxrx583j69KnwRRxJSUns3LmT8PBwGjRowNKlS2nWrBm3b9/mk08+ERbsy8/P5+effzamGf7ll1+MKQZFl5nizWLIkCEMGTKEkJAQ6QINACVKlCA4OJgDBw6wY8cONE0jJydHiEtAQADff/89Q4cOZfz48bRr105Y4OqvkOV5U65cOWbPns3Zs2fx9vYmNzeXvLw8oU6JiYlkZWVRokQJQL/Q5OHDh0KdQL9DNCYmxnjWvQx88MEH/PHHH9SpU0e0ipG7d++aLM51cHAgLi5OoJGepKQkk7GVlZUV9+/fp1ixYsJ+27x4HvrTp0+Fj/Xg36nMDLvHRWZYMiw0A/1iTz8/PwIDA/H29haWherAgQN88cUX/P7770yZMoWSJUuyZs0a6Xd7FPxtb258fX2FffarMHz3ra2tSU1NpXTp0jx48ECwlT57Q1xcnEkbNwSXRSBTnwD6o6h++eUXGjVqZHL9559/Fn7c0syZMxk9ejStWrUycRG5u12n07F//34OHTpEcHAwgLCxuszIOI+meDUq2PcXWFpamgT6QN+B9+jRg1OnTglxSk5ONqZvAf05DiJyZxekevXqQnc7vozw8HBCQ0PJycnBxcWFxMRE5s+fz8aNG4U5eXh4UKVKFb7//nsyMzMJDAyUJihjb29v8rfoAYGM1KpVizlz5uDp6YlOpyM8PJxq1aqRnZ0tdKJ68+bN9O7dm6VLlxongRSm5ObmsnPnTuLi4nj//fdN+lGRWFpacuTIEamDfTLRtGlTQkJCyM7OJjo6mrCwMKEr3UCfSikmJgYHBweKFStGeno6CxYsEOrk4eGBp6cn27dvp2LFisbrDg4OQifT/fz8mDRpEiVKlECn05GZmcmyZcvIyMhg6NChwrwUbx7Z2dkUK1aMIUOGFHkmneg0QQEBAWzfvp3p06djb29PQkICbm5uwnw6dOiAk5MT/v7+HDlyRHjgSnaWLVvG/v37+eijjyhdujR37twRvmq5e/fu9OvXz7hYw5BCXjSXLl0iPDyc6tWrm6SJFrkz5eOPP2bQoEFUrFhRGqcyZcqwdu1avLy8AP156SIXoBpo3ry5MUWtYeKzSZMmZGRkCPst2KVLF+bOnUtGRgbh4eFs377duNBZJDJlWHJwcODChQsmgY4ZM2YQFBQkbMd92bJlWblyJfv372fQoEFMmzZN2oUlsmDYuS1LsMhAtWrVSE1Nxc3NjX79+mFnZ8f7778v1Onw4cMEBgby9OlTypcvT0JCAo6OjkLPaJapTwD44osvmD59OsWLF6dy5cqAfqFJVlaWyZEhIggNDeXRo0dcu3ZNigw4AHPmzGH9+vV4eXnh4OBAfHw8zs7OorWkQ9Z5NMXL0WkyLJF6A2nXrt1fHhT7f0Xv3r1ZtmwZNWrUAPQ716ZMmUJ4eLgQH5nx8PBg27ZtDBw4kIiICABcXV05ePCgYDP5BnSDBw9m+fLljBo1in379hEdHc2aNWukTJsgkvT0dNauXUt0dDSapuHs7MzYsWMpWbIkT58+FXroseLVfP7558TGxtKkSRPOnj1Lx44dmTBhgmjetmquAAAgAElEQVQtANasWUOtWrXo1q2baBXpycnJ4euvv+bkyZNomkbHjh3x9vYutArcHBiCDUUFGkBssCErK0vo2XyvIjs7m7i4ODRNo0aNGmphieJ/RO/evdm3bx+Ojo6FzqOT7Rw62Th69CjR0dHSpDmUjby8PMaOHStlKvuTJ09y/vx5NE2jZcuWUqTHO3/+fJHXDZPYIujRowd9+vShbt26JpOKIp0SExNZuHAh0dHRgD6LkK+vLxUqVBDmBPpx1c6dO43tytnZmY8//lh4urX9+/ebjPVEnJf5IsOGDaNnz54mGZYOHjwoZEdDamoqOp2uyF1pN27cEJqJCvTtfe7cufz666/GNq8ojGzBoqK4ePEiaWlptG3bVmiQplevXoSFhTFixAgiIiI4e/Ysx48f54svvhDmJFOfUJDLly9z//59QJ8+s379+sID7126dOHYsWPCPQzk5eWxdu1aaeaDZEbmeTRF0ahg3/+Q9u3bC9vZd+bMGWbMmGFcWRMTE8OSJUto3bq1EB8DUVFRXLt2zWRLvegzJLy8vNi9ezceHh7GYJ+7uzuRkZHCnBISEpg6darxPEFZBnSXLl3Cz8+PO3fu4OjoSHx8PMHBwdSvX1+ol0zk5eWxZ88e+vXrJ1qlEGlpaaxfv77Qd1D0rl+Z6NGjBxERERQrVoy0tDSGDBkizSKJFi1akJqaSokSJShZsqTxcO9z586JVlO8gqKCDaIPZ/+rcyNEpUqROTCqUPxfoJ7LbzZDhw4lLCxMmkkpxX+G4fmseDMpKgVrfHw81apVEyP0/1PUPILouYWUlBRjesWKFSvy1ltvCXNR/OfIGiySEU9PT8LDw3Fzc+PAgQOAfhf3zp07hTnJ2Ce8yI8//siHH34oWgNvb2+CgoKwsbERrWLkk08+UZsbXgOZ59EURaPSeL6CGzduvPRebm6uGU1Madu2LQcPHuTSpUtomoaTk5Pw3URffvklly9f5saNG3Tq1IkTJ07QsmVLoU6gT5cSFxdn/KEeGRlpks5MBH5+ftKduQjQsGFDNm/ezM8//wzoz6Z8MYXt3x1LS0siIyOlDPb5+vpSs2ZN4uPjmThxInv37qVevXqitaSiePHixh1EdnZ2Upz9YWDv3r2iFd4YikpBYmdnh5OTk9mfO4aJRMPiDRm4cuWKaIUi6devH/v27aNRo0ZqF5bib4FMz+WUlBS+/PJL7t+/T6dOnUyC/uPHj2f16tVCvGTmgw8+YOzYsbi6uppMTLVr106Y082bNwkODub27dsmv0VFpqYEOQPbbdq04cyZM7Rt21aYw4tkZmby1Vdfcfv2bZYtW0ZsbCxxcXG4uLgI9Xr8+DFbtmwp1K5WrlwpzMnd3Z2uXbuydOlS49h90qRJwgO4FhYW3Lx50yTDkqidTgkJCcyZM4erV69Svnx5AB4+fEjdunX54osvhAZGg4ODGTNmjLDPf5OQ8YgeWSlWrBiapvHuu++yZcsWKleuTEpKilAnmfoEKHoOe9asWWzYsAFN04Tu+LW1tcXT05M2bdqYZHWZPn26MKf27dvzzTff4OHhYZJxTS1CNUXmeTRF0ahg3yvw9vZ+6T3RKafKlStHq1atjOdtZGZmCu2QTp8+zb59+/D09GT+/PmMHTtW6HZ6A7NmzWLKlCnExcXRsWNHSpQoITwlj8wDupycHPLz89HpdEID2jLz4YcfcvToUenSLd66dYvVq1dz4sQJXF1d6dKlyyv7sL8jiYmJJoGiF/8WOdCsXLky6enp3Lp1SwVp/4LHjx9z8eJF4+TYiRMnaNKkifHsor/75EJAQIBohSKRMTCqUPxfItNz2c/PjypVqtCuXTt27NjBuXPnWLFiBVZWVty+fVuIk+wYFr/t2LHDeE2n0wkN9k2ePJlu3brh6ekpzXk3IFdg28CuXbsIDQ3FxsbGOEEsOmPCvHnzsLe3Nz4HK1asyJQpU4QH+8aPH0/NmjVp2bKlNO2qTp06VKtWjcGDBxMSEkKZMmWkmFycNGkSAwcO5P333zcuVBJ1Dtb06dMZMGAAYWFhxnPj8/PzOXDgADNmzODbb781u1NkZCQtWrTg9OnTxvG42mX7amQLFsnMxIkTSU9PZ+rUqcybN4+0tDTh6chl6hNAf2TRO++8Y3ItKSmJUaNGodPpOHHihCAzqFGjhrGdy8LSpUuN/4rOziMzMs+jKYpGBftewcmTJ0UrFMnx48dZsGABjx49ApCiQypWrBhWVlbodDpycnKoUKGCMZ2ESGrUqMHu3buJj49H0zSqV68ufPAk64Du+PHjzJkzh/r165Ofn09MTAz+/v7Cf4DKxtatW6VMt2hYgGBtbU1qaiqlS5eW4jsoEwMGDHjl3yI5ffo0c+fOxdLSkpMnT3L58mXWrl0rfHGCjDx8+JDw8HDjGSU+Pj7MmDGD7du307dv3799sO+vzhMWOVFtIDk5md9++w2dTscHH3ygUk4p/iuR6bl869YtVq1aBUDnzp2ZP38+n376KevWrRPi8yYgY1qn/Px8Ro8eLVqjEDIFtg3ImDHh+vXrBAYGEhUVBYCNjQ35+fmCreDp06f4+/uL1jDBwsKCSZMmsXv3bgYMGEBwcLAUKXXbtm3LoUOH+O2334RnWEpNTaVXr14m1ywsLHB3dyc4OFiI0+3bt9m9ezfXr19nzpw51K1bl+fPnwtxeVOQLVgkM+XLl8fOzg47Ozs2btwIQGxsrFAnmfoE0B+j9NtvvzFv3jwqV64MQMeOHaWY2xZ9xFNRqEWor4fM82iKolHBvlfw559/8ujRo0L5jX/44QcqVaokbAv00qVLWbFiBU5OTsZVXKKxsbEhMzOTRo0aMXPmTOzt7aUIYE2cOJGVK1dSs2bNQtdEUXBAB/8+c1E0QUFB7Ny5k+rVqwP6cxHGjBmjgn0vIOPkAUC1atVITU3Fzc2Nfv36YWdnZ2xjCj0yDjANrFq1ij179jBq1CgAGjRoQEJCgmArOUlMTDQG+gBKly7N3bt3sbW1Fb7rXga+/vrrl94TvSsF/r2wpF69emiaphaWKP5rkem5nJ2dbXyv0+nw8/MjMDAQb29vk7SLin+jaRp79uzh1q1bTJ06lTt37vDw4UMaN24szMnJyYmYmBgcHR2FORSFTIFtAzJmTLC2tjb5OysrS4rdau+99x6JiYlUqFBBtIoRQ7l4eXlRqVIlhg8f/tIzf81N2bJl6dChg2gNypQpw8GDB+nZs6cxEKppGgcOHBB2FMe4ceMYN24cH330Eb169eL3338nKSmJbt268f777xMUFCTES0YMZ1DKFiySmalTpxbaJVrUNXMjS58A+u/g1atXmTJlCu7u7vTv31+KhRIgb3rfuLg4YmNjcXFxISMjg5ycHMqUKSNaSypknkdTFI0K9r2CZcuW8dlnnxW6Xrp0ab788kthOy5Kly4t9IdmUSxfvhxLS0tmzJhBWFgYaWlpQgNqBoqaLL9586YAk38j64CudOnSxkAf6Cep1EOuMIYVUrLx5ZdfAvpDvhs0aEBaWppU55TIwNatWxk0aBBAoXNcVq9ezfjx40WpAWBvb2/ytwpcFU2tWrWYM2cOnp6e6HQ6wsPDqVatGtnZ2dIsgBGJjLtRCqIWlij+Lsj0XHZwcODChQs0a9bMeG3GjBkEBQURGhoqxEl2AgICePz4Mb///jtTp07FxsaGRYsWCT0f79KlS4SHh1O9enWKFy9uvC76zD6ZAtsGZMyY0LRpU0JCQsjOziY6OpqwsDA6duwozMfA06dP6dWrF40aNTJpVyJ/yxc8V7R169asW7fOuJNHoWfx4sX4+fkxf/58Y6A2MTERR0dHFi9eLMSpe/fu1K5d2xj0/+STT4iMjGTfvn38/vvvQpxkZfLkyYSHhzNkyBA2bdokTbBIRpKTk0lOTiYrK4vY2FjjYoC0tDSePXsm2E4+6taty+bNm1m1ahVDhgwhJydHqI/M6X3Dw8MJDQ0lJycHFxcXEhMTmT9/vnrevIDs82iKwqhg3yu4c+dOkSsnGzZsyN27d83uY1jN1rlzZ7Zv306PHj1MBuQiz+x7++23je99fHyEeRjYtWsX3377LfHx8Sbn46WlpZkEtMxJwdWIJUuWpEWLFib3RNWfwat169YEBwfz0UcfoWka4eHhdO7cWYiTzNy/f5+lS5cSExNjshpeZP7zgmRnZxtXMGdnZ6vDhQuwd+9e4yAlKCjIZJBy8uRJoYMUGxsbkpKSjCvvoqOjsbOzE+YjM4sWLWLt2rX4+/ujaRrOzs5MmzYNCwuLV+5q+zuSlpZGXFycSV9VcLJfBGphieLvhgzP5SVLlhS5snvSpEm4ubmZ3edNIDo6moiICHr37g3AW2+9JXwXpK+vr9DPfxkyBbYNyJgxYdKkSXz99dfY2NiwdOlSOnbsKDzdKejPeHJ1dRWtYYKXl5fJ33Xq1JH2TGJRVKtWjU2bNpGcnMz9+/cBqFSpktBFxEeOHOHatWuMHTuWVatWcf36dZKSkli3bl2hbFl/d54/f86xY8e4d+9ekSn4RWfikIkDBw6wadMmHj58aOzTAezs7Bg5cqRAM3kpVqwYU6dO5ZdffuHChQtCXWRO77t582b27t1rXGBSo0YNkpKSBFvJh8zzaIqiUcG+V/CqFRAiVkc0atTIeGgowPz586U5RPTmzZuEhISQkJBAbm6u8bqolaatWrXi3Xffxd/f3+SwUFtbW+rUqSPEyVB/L0NU/b3Yrgqu4tTpdAwfPlyIl6z4+vrSo0cPrl27xpdffsmOHTuoWrWqaC0pz/KUjYKpkl5MmyQ6jdLUqVMZNWoUd+7c4ZNPPiE+Pl7YeRsyk5eXx6FDh5gxY0aR92XYJS0Lhw8fJjAwkKdPn1K+fHkSEhJwdHQUtopSLSxR/N2Q6bn8qoC6qGMJZKd48eIm43YZzlZr3ry5aIW/pGnTpqIVjMiWMcHa2poxY8ZIl8bMENCWgWnTprF06VL69OlT5O9m0btYZaRs2bJSjX/ff/997O3tWbNmDaAPJleuXJmdO3fi5OQk2E4eJk+ezLfffktSUlKhxYoypN2XiSFDhjBkyBBCQkKkPLdWZho1akSNGjWEOsic3tfa2hobGxuTazIcRyUbMs+jKYpGBfteQdmyZbl69Sp169Y1uX716lUhq9BlPjx04sSJuLu707t3byk6x8qVK1O5cmUOHjwI6Lf+ix4EG+ovODgYa2tr+vXrh6Zp7N69u9AZDiK8FK9HSkoKXl5ebN68mUaNGvHBBx8wdOhQ0VpSnuUpGwUnDV6cQBCdy75hw4Zs3ryZn3/+GdAPzEWdtyEzlpaWREZG0q9fP9Eq0hMSEkJ4eDgjRowgIiKCs2fPcvz4cWE+amGJ4u+Gei6/2dSuXZv9+/ejaRp37twhNDSUJk2aCHVKS0tj/fr1XLt2zWSX4ebNmwVayYmMGRPS09NZt24d//rXv9DpdDg7O+Pj44Otra0Qn02bNjFkyJCXnh1fcMGsuRgyZAjASxd1KV4PNzc3Dhw4IOzzC+5Ctre3x93dHXd3d2E+MuLi4oKLiwsBAQHMmjVLtM4bwejRo7lx4wbR0dEAtGjRgpo1awq2kouYmBh8fX2xsLAgMDCQwMBAoqOjKVOmDCEhIUJSbMuc3rdMmTLExcUZxwqRkZFUrFhRqJOMyDyPpigaFex7BT4+Pvj4+DB27FgaNGgAwOXLl1m3bh1ffPGFYDu5sLKyknIL/aVLl5g4cSL5+fmcPn2ay5cvs2vXLvz9/YU5nTlzhh07dhj/HjFiBP3792fw4MHCnBSvjyEwW6pUKe7du8fbb7/NvXv3BFvJeZanbCQmJhonNAq+1zSNhw8filQD9DvG8/Pz0el0JjukFaZ8+OGHHD16lG7duolWkRorKyvKlStHXl4eoN/xvnr1amE+amGJ4u+Gei6/2cycOZPFixfz6NEj+vbtS8eOHZk5c6ZQJ19fX2rWrEl8fDwTJ05k7969xhSxClNkzJjg6+uLra0ts2fPRtM09u3bh6+vL6tWrRLiYzgOpFSpUkI+vyjq168PmO5izc7O5smTJ4V2av7duXHjxkvvpaSkmNGkMB988IHxfVhYmEAT+VGBvtcnIiKCL7/8kvbt2wPw1VdfMXXqVHr16iVWTCIWLFjA2LFjSUtLY+TIkUyaNInQ0FBOnjxJYGCgkLPoZE7vO2vWLKZMmUJcXBwdO3akRIkSQs/2lRXZ59EUhVHBvlfQpk0bFixYwLp161i4cCEA9erVY/78+bRp00awnVy0adOm0EGdMrBo0SLWr1/P1KlTAf15DaJ/qKempnLr1i3effddABISEkhNTRXqpHh9mjZtSmpqKv3798fT05NixYoJDTrIfJanbAwYMKDI9wD9+/c3t44Jx48fZ86cOdSvX5/8/HxiYmLw9/fHxcVFqJeMbN26ldTUVEqUKEHJkiWNqfHOnTsnWk0qihUrhqZpvPvuu2zZsoXKlSsLn/xRKP4OqOfyfwe2trYsWLBAtIYJt27dYvXq1Zw4cQJXV1e6dOkixZlvMiJjxoSbN28as84ANGnSROhZeR9//DGgT7EmG5MmTWL+/PlYW1vj7u5OSkoKn376KSNGjBCtJg2G9JhFpVATPbdw+vRplYZS8b/Ohg0b2LdvnzHw/+jRI0aMGKGCfQXIyMigU6dOgD6LiqFsOnbsKGxhCcib3rdGjRrs3r2b+Ph4NE2jevXqUmSqkw2Z59EURaOCfX9B69atad26tWgN6WnZsiU+Pj5YWFgYJxhlmHzNyckpdBaJyJSZoP/x0rdvX+PKxatXrwrdaaj4zzCklfHw8KB58+akp6dTu3ZtYT4yn+UpGzJOZhgICgpi586dVK9eHYD4+HjGjBmjgn1FsHfvXtEKbwQTJ04kPT2dqVOnMm/ePNLS0vDz8xOtpVD816Oey/8ddOvWjUGDBtG7d+9C57mIwnDmnLW1tTEd1oMHDwRbyYudnZ1UAYd33nnH5GiJlJQUqlSpIsznZek7DYhI42kgLi4OOzs7jh49irOzM7NmzaJv374q2FeAypUrs337dipUqFDonqh2/+DBAypWrEhwcLDRYc6cOWquQ/G/RsEdvmq3b2EKBv9btWplck/02cMypvedOHEiK1euNEkHa7im+Dcyz6MpikYF+17BuXPnaNmy5f/z//k7MHfuXAICAqhXr55U55IUK1aMjIwMYx7hGzdumKyuFkGXLl1o2rQpv/76K5qm0ahRI+HnCSr+GsNK/YK89dZbvPXWW2RmZgpbqa9S470+N27cMDnE/vDhwxw8eBAHBwfGjx8v7MwU0Kd7MwT6AKpVqybkbNg3gcqVK4tWeCMwjE3s7OyEpGxRKP6uqOfyfwfLly9n27ZtrF27li5dujBgwADq1Kkj1KlatWqkpqbi5uZGv379sLOzE3L+juI/wxBUK1WqFO7u7nTo0AGAU6dOCZ1HMKTvTEhI4MKFC3Tu3BmA7777TvhiZ0M6+wsXLtCuXTtKliwp1RyDDHTp0oW7d+8WGewz1KW5mTFjBsnJyaSkpHDgwAHq1avH5cuXhbgo/vuoWrUqq1atol+/fuh0Onbt2vX/tXfvQVXX+R/HX4fLyfBGK+EKsoK1kooXkhKtxEVzRcwLocQ6auulAXXFzW11KWqVzdXQ0lZFc3WE2QRSUFvDzZIG2RF2o3FQGcDyPh7EDcXQBVI4vz8czy9WM03ke87x+Zhh5pzv+Z45r1EH4fv+fl4f+fn5GR3Lrvj6+urSpUs3tBOcPXvW8GYJe6z3PXXq1A3Hjh07ZkAS+2bP19Fwcybrzdb9Q5I0duxYZWVl3bQa4brY2Fjt2LGjFVPZp+joaG3bts3oGDfIz89XamqqTp8+rWeeeUYFBQVKSUkxvBsajuexxx5rdqe+JO7UdzDR0dFau3atvL29dejQIU2dOlXx8fGqqKiQm5ubli5d2uqZrg+RN27cKFdXV0VHR8tqtSonJ0dms1nTpk1r9Uz2rrKyUikpKSovL1dDQ4Pt+N69ew1MZX+qq6v15z//WZWVlXr//fdVXl6uAwcOULUBAHfgm2++UXZ2tjZv3qyuXbtq6tSpGjFihNGxVFxcrNraWg0ZMoTKKTt3vbbs+xh9x/zMmTP11ltv6aGHHpJ0bcXhwoULtX79esMyJSQkqLa2VseOHVNubq5cXFwUExOjnTt3GpYJt6e+vl5RUVEaNWqUSktLtX//foWFhWnQoEH8DIq7Ul1drT/96U/av3+/TCaTBg8erFdffVWdOnUyOprd++9//6u6ujpD/6zsqd73gw8+UFZWlo4dO9ZsVV9tba0CAgLYt+9/2ON1NNwaK/tuoaKiQsHBwTcd9l2/yO/l5WVAMvszfPhwZWRkKCIiwq72JQkLC1P37t1VUFAgq9Wq+Ph42155wJ3gTn3H19DQIG9vb0nSxx9/rPHjx2vmzJlqbGw0rELif+vevlsZYTKZGPbdRGJiokaNGqWysjItX75cGRkZ+tnPfmZ0LLvz2muvaciQIdqyZYuka3sSvPLKK1xoAYA7UFJSon/9619q06aNnnnmGWVmZio3N1crV640NFdISIihn4/bZ/Qw74dUVlbaBn3SteaSM2fOGJhIWrZsmf75z38qMDBQHh4eqqqq0vz58w3NhB82d+5cDRw4UGaz2fbvfuzYsZo1a5bh27vA8T3wwAN65513mh27dOmSQWkci4eHh201d2uzx3rfp556St26dVNycnKzyup27doZ3uJgj+zxOhpujWHfLXBx//Zd/4V30aJFdrfayc/P74ZNRAHcf75b/1NSUqJJkyZJklxdXQ2rBuL/mTt34cIFTZgwQenp6QoODla/fv304osvGh3L7lRVVSk2NlZZWVmSrtVaU4EFALdn06ZNyszMlJ+fnyZPnqywsDCZTCbFxcUZVpEHx5aamqr4+HijY9yge/fuevXVVxUdHS1JysnJUffu3Q3N1KZNm2b7Vnfu3PmmdZWwL7Nnz1ZRUZEsFotGjBghPz8/VVdX6/Lly5oyZYrR8eDgJk+erO3bt//gMdgXe6z39fX1la+vr3bt2iVJzfbTxY3s8Toabo1hH1qEvV6w/uKLL7RixQqdOnVKjY2NtiEkd5YB958uXbro/fffV+fOnVVaWmrbJ+Xbb7+17Q0C++fu7i7p2h2KFotFXl5eslgsBqeyP25uzX/E++abb25ZSw4A+H+nT59Wampqs3qn6/53ZQFwKzt37lRoaKjy8/Ntw77x48fbzQXqJUuWaM2aNUpOTpbValVoaKgWLFhgdCw4oMDAQAUGBio3N1dZWVk6ceKEpk2bppycHL3xxhu2C+vAnbh69aquXLmipqYm1dfX236fqa2ttW2JAfuVlpZmq/c9efKkcnNzdfz4cc2dO9fwet+DBw8qISFBTU1Nys/P16FDh/TBBx8YuurQHnEdzfEw7INTS0xM1Lx58xQUFMQdB8B97vXXX9eiRYtUVVWlRYsWqWPHjpKkwsJCDR061NhwuG0hISGqqalRbGysoqKiZDabNXLkSKNj2Z0RI0bo9ddf1+XLl5WTk6MtW7bo+eefNzoWADiEN95443tfCwoKasUkcHSnT5/W1q1bdeTIESUlJalXr16qr683OpZNu3btGO6hRV2vdfP391eHDh305ptvGpwIjmzdunVavXq1TCaT+vfvbzverl07/frXvzYwGW6HPdf7LlmyRBs2bNDvfvc7SVKfPn20cOFCQzPZI66jOR6Tldu84cQmTJigrVu3Gh0DAHAPWCwWXbp0ST169DA6il368MMPlZeXJ6vVqvDwcDr1AeA2VVZWKiUlReXl5WpoaLAd37t3r4Gp4Miio6O1YMEClZaWas2aNerUqZN69uxp+EpRe60XhXM4c+aMfH19jY4BJ7B48WK9/vrrRsfAHaqoqFBRUZHWrFkjT09P+fn5qaKiQqtWrVL//v3l6upqWLbnn39e2dnZGjdunHbs2CFJzR4DjoqlTnBqo0ePVkZGhmpqalRXV2f7AnD/uZ07x4y+uwzf77vfw69/PfTQQ/Lz8+P7+v9obGzUu+++qzFjxmjlypVatWoVgz4AuAOJiYm2mqLly5drwIABGj9+vMGp4IgiIiKUkJCgmpoadezYUZMnT1bXrl31j3/8QzNmzDAs186dO1VVVaX8/HzbMf6No6Ux6ENLYdDnmAIDAzV16lQFBARoz549SkpKktlsVk5OjuG/n5rNZl2+fFkmk0mS9NVXX+mBBx4wNJM94jqa46HGE06tU6dOSkpK0uLFiyXJtmdfWVmZwckAtLalS5cqKyvrlvuWLVu2jDu57FRwcLBMJlOzv7/rz/m+3pyrq6s+//xzo2MAgMO6cOGCJkyYoPT0dAUHB6tfv3568cUXjY4FB7R7926VlZVp9uzZevfdd3XkyBF9/fXXWrt2rQYPHmxYLnuvFwUAOA97rPeNi4vT9OnTde7cOS1cuFAFBQVKSUkxOpbd4Tqa42HYB6f29ttvKz09Xb1792bPPuA+V1FRoeDg4Jv+kHJ9aOTl5WVAMtyO8vJyoyM4lKFDh2rjxo0aN26cPDw8bMcffPBBA1MBgGNwd3eXJHl4eMhiscjLy0sWi8XgVHBUPXv21MMPP6zVq1dLutY+4+vrq8zMzGZ7ULWmOXPmaM6cOYqOjtaYMWNUWlqqr7/+WiNHjrSLelEAgPP41a9+ZXu8Zs0aA5P8v7CwMHXv3l0FBQWyWq2Kj49Xt27djI5ld7iO5ngY9sGpeXt7q0+fPkbHAGAHGBbhfnL9rsSUlBRWQALAHQoJCVFNTY1iY2MVFRUls9mskSNHGh0LDiwxMdH2+OGHH9bYsWMNrTCLiIhQjx49mtWL7ty5U9u3b1dpaalhuQAAzs2e6n39/PyaDSJxI66jOR6T9VbrMAEHt3LlSl25ckWjRo1q1r386KOPGpgKAAAAgCOwWCy6dFd6BUEAAA2GSURBVOmSevToYXQUoEVdrxft1auXrV50xowZGjx4sGErDgEAaA1ffPGFVqxYoVOnTqmxsdF2cyz7z8HRMeyDUwsPD7/hmMlk0t69ew1IAwAAAMBe1dXV3fJ1qpDxY+Xn5yssLMzoGDeIiYlRVlaWpGv1ojNnzlRhYaGWLl1qcDIAAO6dX/7yl5o3b56CgoKabftkTysPgR+DGk84tby8PKMjAAAAAHAAwcHBturj66hCxt04e/asfvrTnyo1NdU27EtKSlJycrLBya6xt3pRAABaQ4cOHRQREWF0DKDFsbIP94Xq6mo1NDTYnvv4+BiYBgAAAADg7KZOnarz58/rwoULWrBggXr37q2XX35ZO3bsMDoaAAD3rbS0NJnNZkVERDTb9okWBzg6hn1waoWFhVq4cKGqq6vl4uKiK1euyNPTkw5mAAAAAMA9V19fr6ioKI0aNUqlpaXav3+/wsLCNGjQIMXGxhqazV7rRQEAuJd27dqlpKQk1dfXSxItDnAaLj98CuC4UlJStHnzZj366KMqKSnRokWLFBMTY3QsAAAAAICTmzt3rrKzs2U2mzVnzhylpqbK399fs2bNsl1gNMLZs2clSampqbZjSUlJRsUBAKBVvf3220pPT1dpaanKyspUXl7OoA9OgT374PQCAgJ09epVmUwmTZw4UZMmTTI6EgAAAADAyc2ePVtFRUWyWCwaMWKE/Pz8VF1drcuXL2vKlCmG5VqwYIGtXvTvf/+7evfurUOHDhmWBwCA1uTt7a0+ffoYHQNocazsg1Nzc7s2z+7cubPy8vJUUVFhu4sRAAAAAIB7JTAwUFOnTlVAQID27NmjpKQkmc1m5eTkaOzYsYblSktL09atW9WhQwedPHlSKSkpOn78uObOnauMjAzDcgEA0BpCQ0OVkpKi0tJSffXVV7YvwNGxsg9ObcqUKbp48aISEhI0f/581dbWKjEx0ehYAAAAAID7xPXBnr+/vzp06KA333zT0Dxz587VwIEDbfWi0rWMs2bNYn97AIDT+/DDDyVJu3fvth0zmUzau3evUZGAFmGyWq1Wo0MA90JTU5MKCgrYcBwAAAAAYBfOnDkjX19fQzNUVFSoqKhIa9askaenp/z8/FRRUaFVq1apf//+cnV1NTQfAAAA7hzDPji1F154QZmZmUbHAAAAAADArsTExCgrK0snTpzQtGnTNGjQIJWUlGjXrl1GRwMA4J6rrq5WQ0OD7bmPj4+BaYC7R40nnFpQUJAOHjyovn37Gh0FAAAAAAC7YW/1ogAAtIbCwkItXLhQ1dXVcnFx0ZUrV+Tp6UmVNRweK/vg1MaNG6cvv/xS3bp1k4eHh+34tm3bDEwFAAAAAID9sId6UQAAWkNUVJRWrFih3/72t9q+fbu2bt0qi8WiefPmGR0NuCus7INTS0xMNDoCAAAAAAB2jUEfAOB+EhAQoKtXr8pkMmnixImaNGmS0ZGAu8awD07tySefNDoCAAAAAAAAAMAOuLldG4l07txZeXl58vX11dmzZw1OBdw9hn1warW1tdqwYYPKysqabbianp5uYCoAAAAAAAAAQGubMmWKLl68qISEBM2fP1+1tbW0w8EpuBgdALiXEhMT5eLiohMnTmjixIlydXVV3759jY4FAAAAAAAAAGhFTU1Nat++vTp27Ki+ffvqk08+UVFRkcaMGWN0NOCuMeyDUzt58qTmzZunNm3aaPTo0Vq/fr0OHz5sdCwAAAAAAAAAQCtycXFRamqq0TGAe4JhH5ya2WyWJLm7u6umpkbu7u50MAMAAAAAAADAfSgoKEgHDx40OgbQ4tizD07N399fNTU1eu655xQTE6P27durZ8+eRscCAAAAAAAAALSy4uJiZWRkqFu3bvLw8LAd37Ztm4GpgLtnslqtVqNDAK2huLhYtbW1GjJkiFxdXY2OAwAAAAAAAABoRf/+979vevzJJ59s5SRAy2LYB6d3/vx5lZSUyGQyqW/fvvrJT35idCQAAAAAAAAAAIAWQY0nnNqePXuUlJSkoKAgNTU1qby8XMnJyRo+fLjR0QAAAAAAAAAArai2tlYbNmxQWVmZGhoabMfT09MNTAXcPYZ9cGrvvPOOMjMzFRAQIEk6ceKE4uPjGfYBAAAAAAAAwH0mMTFRjzzyiE6cOKGEhARlZ2erd+/eRscC7pqL0QGAe6ljx462QZ8k+fv7y9PT08BEAAAAAAAAAAAjnDx5UvPmzVObNm00evRorV+/XocPHzY6FnDXGPbBqT399NNKTU3Vf/7zH507d07r1q3Ts88+q7q6OtXV1RkdDwAAAAAAAADQSsxmsyTJ3d1dNTU1cnd319mzZw1OBdw9ajzh1FavXi1JWrVqVbPjb731lkwmk8rKyoyIBQAAAAAAAABoZf7+/qqpqdFzzz2nmJgYtW/fXj179jQ6FnDXTFar1Wp0CAAAAAAAAAAAgNZSXFys2tpaDRkyRK6urkbHAe4KK/sAAAAAAAAAAMB94fz58yopKZHJZFK/fv0Y9MEpsGcfAAAAAAAAAABwenv27FFERIT+9re/KS0tTZGRkfr000+NjgXcNWo8AQAAAAAAAACA04uIiNDatWsVEBAgSTpx4oTi4+O1e/dug5MBd4eVfQAAAAAAAAAAwOl17NjRNuiTJH9/f3l6ehqYCGgZrOwDAAAAAAAAAABOb/Xq1XJ1dVV0dLSsVqtycnJkNpsVGxsrSXrwwQcNTgj8OAz7AAAAAAAAAACA03vssce+9zWTyaSysrJWTAO0HIZ9AAAAAAAAAAAAgINizz4AAAAAAAAAAADAQTHsAwAAAAAAAAAAABwUwz4AAAAAAAAAAADAQTHsAwAAAIAWFB4erv3799vl5wUGBurkyZM/6nPu5r0AAAAAgHuHYR8AAAAA/IDWHuABAAAAAHC7GPYBAAAAAAAAAAAADophHwAAAADcwiuvvCKLxaK4uDgFBwdrw4YN2rt3ryIjIxUSEqLJkyfr6NGjN33v0aNHFR4ero8++kiSVFVVpd/85jcKDQ1VeHi40tPTbef+5S9/UUJCgn7/+98rODhYkZGROnTo0B1lPXjwoGJiYhQSEqKnn35aixcv1rffftvsnPz8fA0bNkwDBw7UsmXL1NTUZHtt27ZtioiI0BNPPKHp06frzJkzd/T5AAAAAIDWx7APAAAAAG4hJSVFPj4+WrdunQ4cOKDhw4dr/vz5SkxMVGFhoYYMGaK4uLgbhmqlpaWaPn26kpKSFBkZqaamJsXHxyswMFD79u1TWlqa0tLSVFBQYHtPXl6eIiMjVVxcrPDwcCUnJ99RVhcXF/3hD39QUVGRMjMzVVhYqC1btjQ755NPPlF2dra2b9+uvLw8ZWdnS5I+/fRTrV+/XqtXr1ZhYaEGDBig+fPn/8g/NQAAAABAa2HYBwAAAAB3IDc3V2FhYXrqqafk7u6u6dOnq76+XgcOHLCdU1xcrPj4eC1dulS/+MUvJEmHDh3S+fPnNWfOHJnNZvn5+WnixInKzc21vW/AgAEKCwuTq6urxo4dq/Ly8jvKFhQUpP79+8vNzU1du3ZVTEyMPv/882bnzJw5U56envLx8dGUKVO0a9cuSVJmZqZeeuklPfLII3Jzc1NcXJzKyspY3QcAAAAAds7N6AAAAAAA4EjOnTsnHx8f23MXFxd16dJFVVVVtmOZmZl64oknFBoaajt25swZnTt3TiEhIbZjjY2NzZ57eXnZHrdp00YNDQ26evWq3Nxu71e348ePa+nSpTp8+LDq6urU2Nio3r17NzunS5cutse+vr46d+6cJMlisWjJkiVatmyZ7XWr1aqqqir5+vre1ucDAAAAAFofwz4AAAAAuAPe3t46cuSI7bnValVlZaU6d+5sO7Zo0SJt2LBBS5YsUWJioqRrQ7auXbtqz5499yzbH//4R/Xq1UsrVqxQu3bttHnzZn388cfNzqmsrNTPf/5zSdcGfN7e3rZ8cXFxGjNmzD3LBwAAAABoedR4AgAAAMAP8PLy0unTpyVJERERys/PV2Fhoa5cuaJNmzbJbDYrODjYdn7btm3117/+VcXFxVq+fLkkqW/fvmrXrp3ee+891dfXq7GxUUeOHNHBgwdbLOfly5fVtm1btW3bVkePHlVGRsYN52zcuFEXL15UZWWl0tPTNWrUKEnSCy+8oPfee09ffvmlJKm2tla7d+9usWwAAAAAgHuDYR8AAAAA/ICXXnpJqampCgkJ0WeffaaUlBQlJycrNDRUn332mdatWyez2dzsPR06dNCmTZu0b98+rVy5Uq6urkpNTVV5ebmGDRum0NBQvfbaa7p06VKL5VywYIF27dqlxx9/XElJSbZB3ncNGzZMUVFRGjdunIYOHaro6GhJ0rPPPqsZM2bo5Zdf1uOPP67Ro0dr3759LZYNAAAAAHBvmKxWq9XoEAAAAAAAAAAAAADuHCv7AAAAAAAAAAAAAAflZnQAAAAAAMD3s1gsioyMvOlrH330kXx8fFo5EQAAAADAnlDjCQAAAAAAAAAAADgoajwBAAAAAAAAAAAAB8WwDwAAAAAAAAAAAHBQDPsAAAAAAAAAAAAAB8WwDwAAAAAAAAAAAHBQDPsAAAAAAAAAAAAAB/V/uE7atqcrDB4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1793.47x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "scores = []\n",
    "for (i, token_label) in enumerate(token_labels):\n",
    "    scores.append({\n",
    "        \"token_label\": token_label,\n",
    "        \"score\": s_scores[i],\n",
    "        \"marker\": 'start'\n",
    "    })\n",
    "    scores.append({\n",
    "        \"token_label\": token_label,\n",
    "        \"score\": e_scores[i],\n",
    "        \"marker\": 'end'\n",
    "    })\n",
    "df = pd.DataFrame(scores)\n",
    "\n",
    "g = sns.catplot(x='token_label',\n",
    "                y='score',\n",
    "                hue='marker',\n",
    "                data=df,\n",
    "                kind='bar',\n",
    "                height=6,\n",
    "                aspect=4)\n",
    "\n",
    "g.set_xticklabels(g.ax.get_xticklabels(), rotation=90, ha='center')\n",
    "\n",
    "g.ax.grid(True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 总结"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `BERT` 是怎么用 `Transformer` 的？\n",
    "\n",
    "- `BERT_BASE:L=12,H=768,A=12`:层的数量为 `L`，隐藏层的维度为 `H`，自注意头的个数为 `A`，前馈过滤器的维度为 `4H`        \n",
    "- `BERT` 的输入向量表示，比 `Transformer` 多了 `Segment Embeddings`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `ELMo` 和 `BERT `\n",
    "- 从网络结构以及最后的实验效果来看，`BERT` 比 `ELMo` 效果好主要集中在以下几点原因：\n",
    "    - `LSTM` 抽取特征的能力远弱于 `Transformer`\n",
    "    - `BERT` 的训练数据以及模型参数均多余 `ELMo`，这也是比较重要的一点\n",
    "    \n",
    "       \n",
    "- `ELMo` 和 `BERT` 的区别是什么？\n",
    "    - `ELMo` 模型是通过语言模型任务得到句子中单词的 `embedding` 表示，以此作为补充的新特征给下游任务使用。这一类预训练的方法被称为 `Feature-based Pre-Training`。\n",
    "    - `BERT` 模型是“基于 `Fine-tuning` 的模式”，这种做法和图像领域基于 `Fine-tuning` 的方式基本一致，下游任务需要将模型改造成 `BERT` 模型，才可利用 `BERT` 模型预训练好的参数。    \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### BERT 的输入和输出\n",
    "- 最底层输入是文本中各个字/词(`token`)的原始词向量，该向量既可以随机初始化，也可以利用 `Word2Vector` 等算法进行预训练以作为初始值\n",
    "    - 英文词汇被进一步切割，划分为更细粒度的 WordPiece，而不是常见的单词\n",
    "    - 中文语料是直接划分成单字\n",
    "- 除此之外还包含文本向量(`Segment Embeddings`)，该向量的取值在模型训练过程中自动学习，用于刻画文本的全局语义信息，并与单字/词的语义信息相融合\n",
    "- 位置向量(`Position Embeddings`)：由于出现在文本不同位置的字/词所携带的语义信息存在差异（比如：“我爱你”和“你爱我”），因此，BERT 模型对不同位置的字/词分别附加一个不同的向量以作区分\n",
    "- 三者加和作为模型输入\n",
    "- 输出是文本中各个字/词融合了全文语义信息后的向量表示\n",
    "\n",
    "- 在做 `Next Sentence Prediction` 任务时，在第一个句子的首部会加上一个`[CLS] token`，在两个句子中间以及最后一个句子的尾部会加上一个`[SEP] token`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 如何用 `BERT` 结构怎么做 `fine-tuning`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 多标签分类（不同于多类别）\n",
    "- 输入通过得到 `embedding` 表示之后，再并行连接 `n` 个全连接层，再分别接上 `softmax` 分类；每个全连接层对一个标签分类\n",
    "\n",
    "TODO"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `Transformers`模块中`BERT`的简单运用"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from transformers import BertModel, BertTokenizer\n",
    "\n",
    "model_name = 'bert-base-uncased'\n",
    "tokenizer = BertTokenizer.from_pretrained(model_name)\n",
    "model = BertModel.from_pretrained(model_name)\n",
    "\n",
    "input_text = \"Here is some text to encode\"\n",
    "input_ids = tokenizer.encode(input_text)\n",
    "input_ids = torch.tensor([input_ids])\n",
    "\n",
    "with torch.no_grad():\n",
    "    last_hidden_states = model(input_ids)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `BERT`模型\n",
    "- `BERT`代表`Bidirectional Encoder Representations from Transformers`，BERT模型的主要结构是`Transformer`的编码器\n",
    "- `BERT`是基于大量无标签文本(`Wikipedia-2500M单词+书籍-800M单词`)进行预训练的\n",
    "    - 预训练步骤是`BERT`成功的主要原因，在大量语料上进行训练后，模型开始“理解”语言\n",
    "- BERT是“深度双向”模型，训练时从一个token的左右两边学习信息\n",
    "![](images/bert-bidirectional.png)\n",
    "    - 如上图中的单词`bank`，基于其左边和右边不同的信息，有不同的涵义\n",
    "- 通过在`BERT`的预训练模型上添加上一些额外的输出层进行优调，可以很好的用于一些列`NLP`任务"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `BERT`输入\n",
    "![](images/bert-input.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. `Position Embeddings`：两个句子，“武汉到广州的车票”与“广州到武汉的车票”，`self-attention`是不考虑`token`在句子中的位置，因此无法区别这两句话；因此引入位置编码，句子中的每个位置都有一个独特的位置编码\n",
    "2. `Segment Embeddings`：预训练任务`NSP`，输入文本是句子对，需要预测两句话是否是连续的两句话，因此添加句子编码\n",
    "3. `Token Embeddings`：每个`token`的词表征"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-04-09T13:14:15.556639Z",
     "start_time": "2020-04-09T13:14:15.444441Z"
    }
   },
   "source": [
    "### `BERT`预训练任务\n",
    "![](images/bert-tasks.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {
    "height": "526px",
    "width": "595px"
   },
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "165px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
